Extras - A Roundup of Internet News-Filtering ToolsBy Susan Charkes, Published on September 14, 2000
Susan Charkes is the Systems Librarian for Dechert Price & Rhoads in Philadelphia. Ms. Charkes practiced law in New York City for 7 years before becoming involved in the information professions. She has held positions in technology consulting and electronic publishing, as well as in corporate information services for Warner-Lambert Co. and other organizations. She received a BA in English from The University of Chicago, a JD from Columbia University and an MLS from Rutgers, the State University of New Jersey. Ms. Charkes is Director/Web Manager of the Philadelphia Chapter of Special Libraries Association.
The current awareness product cycle seems to be in the beginning stages, analogous to where search engines were three or four years ago, with a few very bright ideas standing out from a sparse field. In this roundup of brief reviews, I selected products for creating Intranet links that integrate news or current awareness into an internal Web page. In addition to services from West, Lexis and Factiva, I discuss free services from Moreover.com.
Since there are many ways to get news it may help to distinguish some services that are not covered here. For example, numerous news sites enable a user to set up personalized home pages that monitor news by topic, keyword, etc.; some allow users to select specific news sources. These generally require a unique login. Examples abound, from MyYahoo, Excite NewsTracker and MyCNN to Crayon, Individual.com and newsalert.com to name but a few. An Intranet manager can set up links to customized pages, although this would tend to defeat the purpose of having personalized links.
Also not covered are special-interest news websites or topic-specific sections of general news websites. For example, say you want to link to general technology news: it is a no-brainer to create a link to, e.g., Andover news, news.com or ZDNet. Typically, news websites will display current news in full text, with recent stories grouped in hierarchical organizational schemes; many have a substantial searchable archive.
What none of the foregoing services provide is the power to set up highly customized filtering services that can be accessed by a group via links incorporated into a Webpage. This is where the reviewed services play. Each of these acts as a topical news filter. The nearest analog equivalent is a clipping service, but not one that sends you clippings in the mail. This is one that sets up folders for you, then sneaks in and slips new clippings into the appropriate folder. It's still up to you to open the folder and look at the clippings.
West's WestIntraClips service is, as with the other West NetSolutions products, easy to set up, albeit somewhat limited in source scope. A designated administrator sets up entries by creating them using a simple online form. Setting up the entry is a simple matter of creating a Terms & Connectors query in a database. It is not possible to query multiple databases. Only selected Westlaw databases are currently available to IntraClips users. Chief among these is ALLNEWS (the contents of which is entirely provided by DowJones Interactive). Major federal and state caselaw databases are included, as well as the Congressional Record (CR); a few other news databases are included but all cover only sources that are also available within ALLNEWS. It would be convenient if the wizard were to provide a direct link to the West database directory to get coverage information for a database.
The default is to remove duplicates but this can be changed for any entry.
The form generates HTML code for a link that can then be incorporated into any Intranet Webpage. Each entry is represented by a unique link.
One nice feature of the service is that the search is run initially at the time the entry is created, and results are immediately available to users. Thereafter the search is updated nightly.
To access the results of any entry, the user clicks on the link. This opens up the results list on the IntraClips server. The first time a user accesses IntraClips, the Westlaw login and password are required (assuming no IP authentication has been set up); thereafter, a cookie will store this data and access will be seamless. The headline, citation and first several lines of each story are displayed. Clicking on a headline will take the user to the main Westlaw server to view the story; the user's Westlaw login and password are required here as well.
The clip service runs a search every night to produce the results list. The results are limited to 10 items. This is a rolling list, as more results are added the older ones are dropped. Within the results list, items are sorted to put newly loaded documents at the top and the remainder (if any) are sorted in reverse order of the publication date. The 10-document limit really forces you to create specific searches or risk missing stories of interest. The limit effectively prevents you from doing some kinds of searches that cannot be made more specific (e.g. a list of the table of contents of the Federal Register, which typically has more than 10 entries).
Email notification of new results is straightforward. For each IntraClip, a daily notification is sent if there are new results. If there are no new results, no notification is sent. The email itself contains the Clip’s link, so with one click, the recipient can access the results. However, multiple individual email addresses are not possible; the administrator has to set up a local solution such as a distributions list aliased to an email address.
The value of the IntraClip product is hard to characterize. Using the regular West IntranetToolkit, an Intranet administrator can create a search link that when clicked on will produce content that (a) is more current, (b) can draw from more sources (or (c) can draw from fewer sources) and (d) is not limited in number of results. Why, then, use a WestIntraClip? Having puzzled over this for some time I think the main reason is cost. If there is a group of users all needing the same information (the same noncurrent, possibly noncomplete information, I might add) then the IntraClip is a much less costly solution, because it does the search once. Otherwise, your users are probably better served by a standard link.
Lexis Trackers have been around for 4 years, with new features continuing to be added since then. Each Tracker consists of a Lexis search; any standard Lexis search statement can be applied; most Lexis sources are available, and sources may be combined. Trackers can therefore be created to cover all sorts of topical news groupings, such as "our clients" or "biotechnology". A Lexis data analyst, not a local administrator; sets up the Trackers and provides the URL, as well as making any changes to them after implementation. A Tracker is a separate product from a Lexis subscription and so is priced separately.
Each search is run daily overnight. To use the Tracker, a separate Tracker id/password is required (IP authentication is available). Clicking on a hyperlink to a Tracker takes the user to a separate server where the Tracker results are displayed. The Tracker results for any id may be viewed by date or by Topic. The screen is divided into a left-hand sidebar and the main results area. The sidebar lists the dates or topics available, together with the number of stories within each. The user can toggle back and forth between date and topic view, select from the individual dates or topics. Clicking on the option changes the result set in the screen. This is a convenient model for viewing stories; the the one wrinkle in the model is that a story is assigned to a date based on the day it is loaded into the system, not the date of the story - this can be confusing when browsing and very confusing when trying to find a specific story. A planned search function, which will be added to Trackers soon, should address this problem.
There is no limit on the number of results per day; all are displayed. Results are archived for 14 days (This will be expanded to 90 days in the near future.) As part of the Tracker service, Lexis staff remove duplicate stories from each day's results.
The headline and citation as well as number of words, are displayed in the results list. Clicking on the headline takes one to the full story, still on the Tracker server not the main Lexis-Nexis server. The full story displays the search terms in bold, which I found to be distracting, revealing the inner workings of the Tracker to the average user is probably not necessary especially since it cannot be modified without calling customer service anyway.
No email notification is available.
I found the screen refresh between result sets, between toggles and between the full story and the result set all to be quite fast, no more than a few seconds on a T1 line. Refresh rate even increases after you have clicked on a screen because it is cached.
Like the WestIntraClips product, the Lexis Tracker's primary value seems to be that it eliminates multiple identical searches. The Tracker provides more comparative value than the IntraClip because of its breadth of sources and larger archive.
DowJones Custom Clips service is similar to West's CustomClips in ease of use and available sources but similar to Lexis Trackers in its display. It is not, however, a true Intranet linking tool.
This product allows multiple designated Intranet content managers to set up and maintain CustomClips. (Although individuals may set up CustomClips, the Group CustomClip is the appropriate avenue for an Intranet link, and the group portion of the service is what I have described here). Each user may be assigned to a maximum of 100 groups and each group may be administered separately if the organization chooses (there is a 100 group maximum oer organization).
Most (6000 of the 7000 or so sources) DJI content sources can be used for a CustomClip; the exceptions are those sources that are no longer being updated and a few where DJI's content suppliers have not agreed to make the content available for this service.
Each CustomClip is referred to on DJI as a "Folder". There are personal Folders for individuals and Group Folders for groups.
Setting up a CustomClip is done through a multi-step wizard on the DJI Website. The administrator selects one or more sources. The DJI service has sources grouped by industry, topic, or type for ease of combining, or you can select any combination of your own choosing. Selection is done by clicking on a list of sources. Information about each source is available with one click on an info button.
If the CustomClip topic is one of several hundred pre-configured searches, results can be viewed immediately. However for custom searches, results do not appear for 24 hours.
The update schedule for a DJI Custom Clip is its most compelling feature. The Clip's results are continuously updated. Thus, whenever the Clip is accessed, the results are as up to date as if the search had been done afresh. Of course, the actual currency of any Clip depends on the currency of the underlying database. so unless the content provider updates data continuously the Clip may still reflect old news.
Email notification is not available for Group folders, only for the single-individual folders. This is a pity, because the email notification for individual folders is flexible and powerful, offering a choice of headlines (linked to full text) or full text; and also a choice of continuous or digest form of update.
Unfortunately, it is not possible to link directly to a Clip. You can create a link to the "Custom Clips" service, but no to individual news trackers. Thus, I almost hesitate to put this service in the same group as the others. However, because individuals have their own customized page, a member of a group can get to his or her own folders by clicking on a link to DJI. There are also a number of very powerful display-customizing features that DJI offers as part of the individual-user service, which, however I do not cover here, because the point of this review is to focus on making information available to a group.
Note: A higher-end product called Factiva Publisher is apparently where the development dollars are going for Factiva. This just-released product, which I do not cover here, enables the Intranet administrator to build portals to Factiva content including DJI as well as Reuters content that is not on DJI. Unlike CustomClips, it requires that data be maintained on a separate server either provided by Factiva, as an ASP service, or located within your organization. Results are stored on the separate server, so the user is not taken into the full DJI service.
Articles are archived for up to 30 days; however, there is an upper limit of 500 articles per folder, so articles may be deleted even if they are more recent. Results are displayed in a framed format with the list of Folders down the left side, and the results in the main window. Unlike the individual Clips service, Group Folders do not indicate which folders contain new (i.e. unread) headlines. results are sorted in reverse chronological order by date of loading. Each entry in the results list includes the headline, citation, word count, relevance score and first sentence or two. Clicking on the headline brings up the whole story in the main window. A nice feature is the "Format to Print/Save" which puts the article into a new browser window free of the free of the navigational features of the display within the result set.
Whether in the display format or in the print/save format, search terms are highlighted and there is no way to change this (again, individual folders have this formatting option). I find this to be annoying -- with a current awareness search I normally don't want to know how the search was done, just the results please. Only the administrator needs to see the search terms.
DJI Custom Clips is a great current awareness product and a lousy Intranet product. I suppose this is why Factiva has come up with a new product. For DowJones Interactive subscribers with Intranets, Factiva Publisher Factiva needs to get around the value problem.
Moreover.com provides a convenient tool, which it calls a Webfeed, for incorporating news headlines from hundreds of sources into an Intranet. The free service does not offer custom sourcing, although a fee-based custom add-on service is available. However, even in its free incarnation, Moreover provides a stunningly easy method for linking directly to free topical news content.
Moreover draws on public Web news sites. The company boasts "over 1500 sources," grouped into more than 300 categories. The sources run the gamut from giant media sites like dowjones.com and msnbc.com, to specialized business and industry sites like Fiberoptics Online and Defined Contribution News. International news sites are well-represented.
One can preview any Webfeed to see what kinds of stories are showing up. Each Webfeed has a separate URL. This also means that if you don't want the content itself to be integrated into your Intranet page, you can just insert a quick link to the moreover Webfeed.
One note on the generated code: it includes a line "Get these headlines on your site" which links back to Moreover. Since this line makes no sense in the context of an Intranet, I removed it and replaced it with a line stating "Headlines powered by Moreover", linked back to the Moreover site. This is a permissible modification of the code, according to the online knowledge base. Since the "Get these headlines" line is not obvious, this required some detective work to find and replace the script line with my own . I t would be nice if Moreover would make this line optional in the wizard or at least post some instructions on how to modify it.
Each Webfeed is real-time news. Moreover claims that its database is refreshed every 15 minutes. The most current set of headlines is loaded whenever the user refreshes the page. (Or you can add a META tag to have the page refresh itself every 15 minutes.) However, this does not guarantee new headlines every 15 minutes unless the source sites are updated as often. Other than general and financial news resources, few sites are updated more often than daily anyway.
Keyword filtering and source restrictions within a category are possible, although currently one cannot use the wizard to set these up (supposedly it's due to be released soon). Instead, the online knowledge base and the developers area (http://w.moreover.com/dev/custom/) contain instructions on how to do this by modifying the wizard-generated code for any category to use a different URL (it is actually a different server) and attach appropriate parameters. The keyword filter supports Boolean (and/or/not) searching. For example I set up a keyword filter on the Tobacco industry news feed to search for keywords on tobacco litigation such as lawsuit, litigation, court, damages. Webfeeds can also be modified to restrict to certain sources.
The keyword filter looks only at headlines; a similar function (query) can be used to searches the body, but these cannot currently be used within a category. The wizard does not currently support either of these options but complete instructions are found on the Moreover site.
As I noted, help is readily available for this product. In addition to basic help for setting up the Webfeeds there is an online help forum which constitutes a fine knowledge base. The company's staff seem to pay attention to these forums and provide responses within a few days.
There are a few disadvantages to this service. No email notification is available; although with a real-time push product email is somewhat redundant. Webfeeds do not always remove duplicate stories. Some de-duping is accomplished automatically by checking headlines, but otherwise the service relies on editorial staff to check for duplicates, and Moreover admits that this is difficult to accomplish. Finally, a Webfeed is limited to current stories; there is no set archive. You can play with this a bit by expanding the number of stories displayed at any one time, but the maximum is 20 (again, this may change...).
Moreover is a very useful tool for tracking news from public Websites - and it has the potential to become more powerful and more easy to use, if the promised improvements are implemented.
In some ways this is an unfair comparison because I did not test or review the Webfeeds from custom sources. However, the free version of Moreover's Webfeeds is a clear winner. It is a simple, effective way to integrate current news into an Intranet. Don't expect a Webfeed to substitute for current awareness, though, unless your definition of awareness is limited to the eternal present.
Summary of Significant Pros and Cons for Each Service
Westlaw: IntraClips Lexis: Trackers Factiva: CustomClips Moreover: Webfeeds Pros easy setup; one-click access to results wide scope of search sources; no limit on results continuous update; precision source control continuous update; headlines "pushed" to Intranet; free Cons 10-result maximum; limited sources lack of administrative interface no direct link to results; slow sources limited to free Websites (in public version); no true archive