PrefaceThe Feed Discussion Plus plugin was created to address requests from users in my intranet community. My research discovered that many feeds do not adhere to feed coding standards, some mix more than one standard, and the standards themselves are outdated and lacking. I attempted to address all these issues, but there are no guarantees - a specific feed you define with this plugin may not import items at all or only at some times.
I'd like to give credit and thanks to the authors of the original Feed Discussion plugin which I used long ago as a basis for this plugin.
The Purpose
Simply to increase forum engagement by importing relevant information from multiple external sources. You can import feeds from sites that support RSS, Twitter Feeds, and YouTube channels. If your selection of sources, imported material, and caegorization make sense, users will more likely comment on the imported discussions and be engaged i the forum.
Typical example include product forums where product information, announcements, news are imported to allow users to comment on them.
Main Features:
- Support for both ATOM and RSS encoded feeds in compressed and uncompressed form
- Support for YouTube channels and Twitter Feeds
- Filtering of content imported into the Vanilla forum based on specified keywords
- Optional setting of minimal number of words per imported feed item (to avoid teaser feed items)
- Optional limit on the number of items imported from each feed
- Option to show the feed's/site's logo instead of the default "system" as author
- Option to remove linked images from imported feed
- Option to set server window hours (for when feeds are imported)
- Option to disable individual feeds (while preserving their definitions)
- Feed imports can be controlled manually by the admin, cron job, or automatic
- Detailed report of admin initiated import
- Autodiscovery of the feed url when admin enters a host url (that supports auto-feed discovery)
- Individual settings for each feed
Performance and Feed Import Operation
It is important to understand the performance impact of feed imports on response time and scalability. Since Vanilla does not have an independent scheduling mechanism running on a separate thread (PHP is a single-thread application - it cannot split off a task to run separately from a page load), the only automatic internal way to check whether new feed items need to be imported is by interrupting a page load request initiated by one of the forum users, performing the feed import task, and then complete the user request. If the amount of imported feed items is large that specific user may have a longer wait until his/her requested page loads. If the number of feed items to import is small, the delay may be negligible. Fortunately this plugin has several design features that reduces the performance impact.
First, to reduce this impact
the test for new feed items is limited to these events:
- User-initiated request to display a specific discussion (when the user displays a list of discussions no feed-import test occur). The assumption is that there is more browsing through discussion lists than fetching a specific discussion, and thus most of the forum activity won't be affected. As you will see below, there are additional options that reduce the import overhead for each checked feed. Note that the user-initiated process is disabled for mobile users (as to not further slow down users who may be on a slow connection). Furthermore, the admin can disable user-initiated imports through the plugin global configuration option, leaving feed imports to the following two options.
- A cron job can invoke the feed import process. Clearly it is better to schedule a cron process to low activity hours, but either way, a user loading a specific discussion will unlikely be affected.
- Manual feed import initiated by the administrator (in which case the administrator may sense a delay during the import process).
Additionally, in the global configuration options you can set a maximum number of feeds that are checked each time. This reduces the total overhead of the feed checking, especially when you define multiple feed sources. If user-triggered feed import is enabled and you have many defined feeds to import, a low max setting will spread the overhead among multiple users possibly to an unnoticeable amount.
As mentioned above, the administrator can also disable the user initiated method entirely thus limiting imports to the last two options mentioned above (See "Advanced Settings" below). This may be advisable if the number of imported feeds and items is large and their import setting is frequent (e.g. frequent news grabbing site).
Regardless of the three import methods mentioned above, each activated feed has these setting options:
- Import Frequency (e.g. hourly, daily, weekly,every specific weekday).
- Optional window hours when imports are performed. Note that if you specify window hours for user-triggered imports and there are no active users who bring up discussions for display at the permissible window hours, user initiated imports will not occur (e..g You set the import window hours to 03-05 in the morning where all your active users are sound asleep).
- Optional limit on the number of items imported per feed.
- Optional filters on the content of items to be imported.
Once the import process starts, it checks each of the defined feeds to see if it is active and if it has a defined window of import hours. Only if it is active and within the specified window active hours, the feed will be a candidate for import. Thereafter every candidate feed for import is checked whether new feed items are available for import. Such new items will still be subject to any defined filters. The overall process may result in many, few, or no items to import.
There are three performance issues to consider:
- The checking of feeds active/inactive state and import window hours. This is done without looking at external data and is efficient.
- Checking if a feed has new items (This only happens if the tests in #1 above succeeds). This requires loading the feed's data from the website and takes longer. This is where most of the delay occur (as it depends on your internet speed and external servers response time).
- If there are new items to import (namely they also passed filter checks) then the saving of new discussions require Vanilla database access to save each item as a new discussion.
Feed InformationThe configuration screen displays information about the feed. It tells you if the feed is compressed ("gzip" attribute), whether it is an RSS or ATOM feed, and other settings you define for each feed. If the feed overall title is available it will be shown. If not and the website title is defined, it is displayed instead.
cron process
The manual import process requires a Vanilla administrator permission. Since the cron job is external to Vanilla the cron link does not require this permission. To prevent users from invoking the cron link (should they ever figure that out) the global configuration screen let you set up a secret code (saved in the Vanilla config.php file) which you then need to specify in the url cron job you set up. For example, for cron code "mysecret" the cron job should be set to invoke the following link:
yourforum/plugin/feeddiscussionsplus//CheckFeeds/cron/mysecret
Advanced Settings
Imported DIscussion Author
The Global configuration screen let to set up the "Import save userid". This will appear as the author of the imported feed item (default userid is "Feed"). You can use this feature to define such a userid and create a thumbnail that would be used to highlight that the discussion is imported from a feed. If you have not defined such user the System user is used as the author.
Displaying the Feed LogoThere is a forum option that displays the feed's logo. If this option is set the plugin style setting will try to overlay the author's thumbnail with the forum logo. It's a display matter only - the actual author is set in the manner explained above. Note that this forum logo presentation instead of the author's thumbnail is theme-dependent. It may not work for some themes in which case you may want to override the plugin CSS (I recommend using the CSSEdit plugin for that).
Limiting Import EventsAs explained earlier, there are three possible feed import events:
- User-initiated (by displaying a specific discussion) - can be activated/deactivated by the global settings.
- cron-initiated (needs to be set in the global settings and your cron scheduler on your server).
- Admin initiated (manual import).
Diagnosing errorsValidating Feeds
You can use the " Issues ? Run Feed Validator" button on the feed edit screen to run a feed validator on a feed url. The results of the feed validator do not necessarily correlate with this plugin's ability to read feeds, but feeds with major errors would probably not be imported.
Presentation Errors
The plugin was tested on several themes. If the feed information does not display correctly try another theme or use the browser's web inspector to check and correct the error.
What version and source are you using
The plugin title on the admin screens of the feedlist and feed edit shows the plugin version (e.g. Feed Discussions Plus (Version 1.1.2). If you hover the cursor on that title the specific php source file name is displayed.
Error Messages
The plugin attempts to report on possible errors. The error messages include numeric code combinations that reflect the line numbers in the code where the error occurs. Hopefully this will help resolve issues.
Perform Manual Import First
It is advisable to try to perform an admin initiated manual import for any problematic feed just to see what happens through the import reports. To see more detailed reporting turn on "Detailed import Report" in the Global Settings.