CSU Libraries > How Do I > Research > Web Search Engines and Directories on Parade

Web Search Engines and Directories on Parade

Here is an assortment of leading searching engines and directories. Most search engines are a combination of two basic categories. Crawlers use a computer to search for (spider) web pages, keep a crude index of them, and retrieve a list of them when users enter keywords. Directories are lists of web pages sorted by topic and compiled by human editors. In addition, search engines often feature useful extras such as news headlines, Usenet, image, or command searching.

The search engine that works best for you can be the one that best fits your search, one with which you are comfortable, and one that offers few commercial distractions. Below is a summary of assorted search engines. For a more extensive and authoritative search enginecomparison, try Search Engine Showdown.

Search Engines Best Known for Their Crawlers

Google http://www.google.com
Google is powered by its own crawler. It has a clean interface and clear but unobtrusive advertising. It also offers categories (directory) from the Open Directory, image searching, and access to Usenet. It is the most popular search engine at present, but not the only one. Note: if you are used to using AOL search, you are using Google.

Windows Live http://www.live.com/
Was previously MSN Search. Windows Live is now uses its own crawler. Also because it uses a different provider for its crawler, it may pick up hits that Google misses. Advanced search allows for searching by file type and Boolean searches, too.

Alltheweb http://www.alltheweb.com
Similar to Google but uses Overture to provide it's crawler. It also features Open Directory for its categories. Because it uses a different crawler than Google or MSN, it picks up additional items.

AltaVista http://www.altavista.com
An older search engine that uses the Overture crawler and Look Smart for its directory. AltaVista features strong image searching and a command language that performs searches none of the others can touch. These include searches for pages that link a given site, pages hosted on a given site, and finely restricted image searching.

Ask http://www.ask.com
Previously called AskJeeves, Ask uses its own crawler and offers news, maps, driving directions, and weather information.

Yahoo http://www.yahoo.com
Yahoo is both a directory and a combination Google/Overture crawler. The directory was and still is Yahoo's strongpoint. In addition, Yahoo offers news headlines, maps, driving directions, and delayed financial statistics.

Lycos (Terra Lycos) http://www.lycos.com
This was once a very popular search engine. It offers Open Directory categories and Overture for its search engine. At one point it was the search engine that many users found comfortable. If you don't like Alltheweb and want to use another Overture powered engine, Lycos is worth a try.

Search Engines Best Known for Their Directories

Yahoo http://dir.yahoo.com/
This was the original directory based search engine though the size of the net ultimately overwhelmed it. It became very difficult get a place inside its directory structure. Search engines and other features give Yahoo its strength.

Dmoz http://dmoz.org
Dmoz, or Open Directory Project, is the directory most other search engines borrow for their categories. The original Dmoz is ad-free and works surprisingly well for authoritative subject oriented searching.

About http://www.about.com Not a directory or search engine in the traditional sense, but a meta site that leads to articles sorted and edited and updated by human editors. Though some assail the editors' qualifications, the editors tend to do a good job with most categories, especially those lying outside academic subjects. About is hopelessly commercial and pop-ups, pop-under, and banner advertisements mar the site. It also features an ugly stripe that you need to remove via a small link to get a correct URL for your works citeds.

Meta Crawlers -- Search Engines that Search Multiple Engines

Meta crawlers are a good idea when you need a large number of results rather than a few good ones or, as in the case of Kartoo, you need to see the relationships between web sites.

Clusty http://clusty.com/
Previously called Vivismo, Clusty is a meta crawler that groups results by category. Has an uncluttered interface and feels fast.

Kartoo http://www.kartoo.com/
The magician, Kartoo, maps results visually and shows the connections between them. This is especially effective when searching for personal web sites.

Dogpile http://www.dogpile.com
Dogpile is a very old meta crawler that allows users to chose the engines they wish to search. More importantly, Dogpile, works well on older machines and slower connections. Dogpile often goes where other metacrawlers can't.

EHK -- December 2004
ALH -- Updated February 2007