I want to know how search engines work?I mean they can read the text in each site of the net,it can cache it

Can we make one of our own using some script or some server requirement or anything ?

Please tell

I want to know how search engines work?I mean they can read the text in each site of the net,it can cache it

Can we make one of our own using some script or some server requirement or anything ?

Please tell

<{POST_SNAPBACK}>

Different search engines work differently. try a search on google about that and you'll get a lot of explanation im pretty sure.

And yes you can code one on yourself, just like google did. You only need a server, a couple of IT tech guys, KNOW HOW SEARCH ENGINES WORK, programmation knowledge and a couple hundred of thousand dollars.

See?

Google your question "how do search engine work?", clicked I'm feeling lucky and got that

How do search engines work?

But next time, some flamers will probably insult you for asking such a n00b question without even looking around in google or on this forum first. Especially if it's something as easy as "I'm feeling lucky" to find.

Well Just want to tell u that google store the text written in description

Meta tag & if it is not available

then the text of Title tag <title>

Yes google stores the meta info, but it also caches the whole page and indexes the images

Also if you were to start your own search engine good luch (and many in a few years you will start your own web domination scheme, just like google, not many people know just how big their empire is (things like google earth and google labs are just the start))

Does anyone know what programming language is commonly used for crawling? Perl? Python?

According to google job openings (http://www.google.com/support/jobs/bin/topic.py?loc_id=1104&dep_id=1056),

" * BS or MS in Computer Science or equivalent (PhD a plus).

* Several years of software development experience.

* Enthusiasm for solving interesting problems.

* Experience with Unix/Linux or Windows environments, C++ development, distributed systems, machine learning, information retrieval, network programming and/or developing large software systems a plus. "