Hi,
I created a Moodle Rollyo and noticed that very few search results from Moodle forums are shown. Rollyo works with Yahoo Search and the "Open to Google" option does not open to other search engines than Google and Altavista.
Why is the search restricted to these two search engines?
I propose to add Yahoo search and probably MSN search that Moodle will be more widely searchable.
Urs
Martin, I hope this List of Robot Agent Strings will help. Many facts are listed on the specific Robot page linked from that list.
Thank you very much for you fast response.
Urs
Another possibility: log someone on as a Guest when they access /robots.txt. This would probably require less effort than maintaining a list of User-Agents.
log someone on as a Guest when they access /robots.txt.
That wouldn't work without session tracking and would be problematic on many shared host servers that won't execute a php script in a file with a txt extension. We do have session tracking without using cookies (search engines won't accept cookies will they) but it is not standard yet and could do with some thorough testing.
