Language Selection

English French German Italian Portuguese Spanish

PhpDig excels at small Web site indexing

Filed under
HowTos

Webmasters looking to provide search capabilities for their site would do well to try out PhpDig, a Web spider and search engine written in PHP with a MySQL backend. There are other open source search engines, all of which have their own advantages. PhpDig just happens to suit the needs of my Information Technology for Greenhouses and Horticulture site. Here's how I got it working.

Webmasters with small sites know the problem of providing useful site search capabilities. Typically, visitors enter keywords in a search box and the search engine returns a ranked list of pages related to the query. This is a useful service -- provided the visitor can tune the search and the results returned are reliable and relevant.

Some Webmasters rely on Google for this service. A listing in Google or another mainstream engine is a must-have in practical terms, so it is easy enough to piggyback on the main engine with a site-specific search, provided Google understands your site and keeps coming back for updates -- but this isn't always the case.

Large search engines boast of indexing of billions of pages, but we are only interested in digesting a hundred pages or so. We need them indexed on a regular basis, daily or at least more often than Google might do it.

It is also important to know if our site is responding correctly by providing public pages, hiding private pages, and following links correctly. Since Google uses algorithms that it doesn't share, we have no way of predicting the indexing results or doing any testing in advance. Advance testing is useful if, for example, you have private files that you want to be sure will not be indexed, but you are relying on your robots.txt file to deny access to bots. If we make a spelling mistake in robots.txt, our private pages could go in Google's cache for the world to read. We also need to control what words are indexed and customize our own search and result pages.

Enter PhpDig.

More in Tux Machines

Getting OpenStack Ready for the Enterprise

OpenStack is gaining popularity as the cloud platform of choice for IT organizations. This was reflected in a 2013 IDG survey that found as much as 64 percent of IT managers including OpenStack in their technology roadmap. In the current fast-paced IT market, the massive scalability and flexible, modular architecture of OpenStack can help give organizations the agility they need. Read more

Open source projects that warrant data center managers' attention

When you're making the case to a data center manager about tech that is worthy of her consideration, make sure these three open source options are on your list. Read more

Open source and Made in Italy: Arduino are circuit boards with a sense of style

One of the more surprising applications has been the natural marriage between the Arduino board and Lego. Once seen only as a child's building block toy, Lego is finding startling utility as an instant mechanical prototype maker for Arduino ideas. Read more

11 Useful Utilities To Supercharge Your Ubuntu Experience

Whether you’re a relative novice or a seasoned pro, we all want to get the most from our operating system. Ubuntu, like most modern OSes, has more to offer than what is presented at first blush. From tweaking and refining the look, behaviour and performance of the Unity desktop to performing system maintenance, there are a huge array of useful utilities and apps that can help tune Ubuntu to meet your needs in no time. Read more