Writing Moodle's Global Search

Re: Writing Moodle's Global Search

by Daniel Neis Araujo -
Number of replies: 21
Picture of Core developers Picture of Plugin developers Picture of Translators

Hello,


you may take a look at https://tracker.moodle.org/browse/MDL-31989

and try the newest code on top of 2.9dev that is mentioned on the last comment.


Kind regards,

Daniel

In reply to Daniel Neis Araujo

Re: Writing Moodle's Global Search

by Mae Church -

I am very interested in the status of the Global Search, I think it is MDL-31989.  I need to provide a capability to our users and would prefer to use a moodle capability instead of an external source.  I have read the documentation but am not clear if this function is available.  Any update would be greatly appreciated.  

In reply to Mae Church

Re: Writing Moodle's Global Search

by Daniel Neis Araujo -
Picture of Core developers Picture of Plugin developers Picture of Translators

Hello,


this feature is not fully available on Moodle,

but if you manage to apply the patch following links on the tracker,

the result is pretty usable, altough it lacks various capabilities checks

to verify if user can really access the content searched.

Also, Solr is a little complicated to get running and is the only

search engine supported right now, so I've started to write some code

to get elastic search integrated too:

https://github.com/danielneis/moodle/commit/d24d6e165e440c69240a3f2e465c65ba3e60a16a


Finally, using only Moodle and a relational database to implement

this feature should not be the best option even if you use something like

sphinx, the features to index content of different file types and decouple

from the main moodle database and webservice are the mais advantages of

this approach with solr/elasticsearch.


If you are interested in funding further development of these features, let us know =)


Kind regards,

Daniel

In reply to Daniel Neis Araujo

Re: Writing Moodle's Global Search

by Mae Church -


Thank you I appreciate your quick and thorough response.  I am not sure if I can fund further development but I will look into that.

 

Is there another recommended approach while we wait for this approach to be completed?

We are on 2.9

In reply to Daniel Neis Araujo

Re: Writing Moodle's Global Search

by Nathalie Georgiou -

Hello !

I would like to know if anything has been finished on this issue? I am very interested in a Global Search on Moodle. 

I'm currently developing a training portal for my company, and being able to look for documents which contain certain key words would make the entire training process a lot easier.

Can you please let me know if there is any such search available on Moodle?

Thanks!
Nathalie

In reply to Nathalie Georgiou

Re: Writing Moodle's Global Search

by Marcus Green -
Picture of Core developers Picture of Particularly helpful Moodlers Picture of Plugin developers Picture of Testers

Global search is not available in core Moodle. All discussions I have read involve additional software including a server side setup called Solr 

https://lucene.apache.org/solr/

which is java based. 

In reply to Marcus Green

Re: Writing Moodle's Global Search

by Nathalie Georgiou -

Thank you so much for your answer Marcus!

I will check this option more. 

If anyone has used it and has feedback, feel free to share with me. 

Thanks!

Nathalie

In reply to Nathalie Georgiou

Re: Writing Moodle's Global Search

by Daniel Neis Araujo -
Picture of Core developers Picture of Plugin developers Picture of Translators

Hello,


tha "Global Search" feature was integrated in Moodle last week (https://moodle.org/mod/forum/discuss.php?d=328852) and yes, it needs an external "indexing and searching" service.


Kind regards,

Daniel

In reply to Daniel Neis Araujo

Re: Writing Moodle's Global Search

by Nathalie Georgiou -

Great! It looks like I have asked just at the perfect time! Thanks for the answer Daniel

Best,
Nathalie

In reply to Daniel Neis Araujo

Re: Writing Moodle's Global Search

by Nathalie Georgiou -

Sorry to get back to you again, but I'm hoping you can help.
My system team has now installed the tool, but how do I make it visible on the entire website?



Thanks!
Best,
Nathalie

In reply to Nathalie Georgiou

Re: Writing Moodle's Global Search

by David Monllaó -

Hi Nathalie,

You can follow https://docs.moodle.org/dev/Global_search, this might slightly change during the next weeks though.

In reply to David Monllaó

Re: Writing Moodle's Global Search

by Nathalie Georgiou -

Thank you David for your answer, I will follow that page for further developments.

Best,

Nathalie

In reply to Nathalie Georgiou

Re: Writing Moodle's Global Search

by Daniel Neis Araujo -
Picture of Core developers Picture of Plugin developers Picture of Translators

Hello,


if anyone is interested, I've developed a search engine plugin to use Elasticsearch:

https://github.com/danielneis/moodle-search_elasticsearch


With elasticsearch you do not need any php extensions, it uses curl to access the rest api of elasticsearch.

Just point your moodle to the url and port of elasticsearch and it works.


Kind regards,

Daniel


In reply to Daniel Neis Araujo

Re: Writing Moodle's Global Search

by David Monllaó -

I'm sorry Daniel but admins with interest in keeping their site contents secured should not use your plugin. You are making all indexed contents available to everyone and you are not filtering the index contents by the context where the user has access, also all filters are ignored and this solution will not scale properly. Please do not recommend this for any production site.

We included Elastic search as a proposal for a GSOC project (https://docs.moodle.org/dev/Projects_for_new_developers#Elastic_search) where we aim to have a secure and scalable elasticsearch plugin for Moodle 3.2. Moodle 3.1 will come with support for Solr and, if we have time to complete it, support for a "local db table" search engine (only for small moodle sites - MDL-53226)

Average of ratings: Useful (1)
In reply to David Monllaó

Re: Writing Moodle's Global Search

by Daniel Neis Araujo -
Picture of Core developers Picture of Plugin developers Picture of Translators

Hello, David


thanks for your review.

The plugin is not really ready for production and that is why it is "only" on the Github and not yet sent to Plugins directory.

I've followed the instructions on docs and thought that calling the "check access" function was enough to check by cotext as you can see at:
https://github.com/danielneis/moodle-search_elasticsearch/blob/master/classes/engine.php#L91-L99


Also, I've found that in the docs it says to use \core_search\document::set_data_from_engine , but instead the Solr plugin uses $this->to_document and it worked for me too, because when I try to use the method on the documentation, Moodle says it should not be called statically.


Finally, I would like to know why you think this plugin is not scalable.


Kind regards,

Daniel


In reply to Daniel Neis Araujo

Re: Writing Moodle's Global Search

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

Hi Daniel -

Since your new plugin is still under development, can I suggest you make the following changes to your "version.php" file?

  • Add a line for "maturity" and set it to beta or alpha. So:
    • $plugin->maturity = MATURITY_ALPHA;
  • Add a line for "release" for a more readable version description. So something like:
    • $plugin->release = '0.1 - Alpha';
  • Change the "requires" to be the Moodle 3.1 version. This will prevent it from being installed on any version prior to 3.1. So:
    • $plugin->requires = 2016031000;

See the details on version.php for more info.

These changes will help ensure  that a Moodle admin will be aware of its current development status.

mike

Average of ratings: Useful (2)
In reply to Mike Churchward

Re: Writing Moodle's Global Search

by Daniel Neis Araujo -
Picture of Core developers Picture of Plugin developers Picture of Translators
Hello, Mike

thanks for the suggestions!
I've made those changes and they are now on the repo.

Kind regards,
Daniel
In reply to Daniel Neis Araujo

Re: Writing Moodle's Global Search

by Eric Merrill -
Picture of Core developers Picture of Moodle HQ Picture of Peer reviewers Picture of Plugin developers Picture of Testers

The docs don't yet fully represent the current state of the API. When I get done with file indexing, I'm going to work on updating them. I would look at engine and the mock stuff (tests/fixtures) to see what the bare minimum is.

In reply to Daniel Neis Araujo

Re: Writing Moodle's Global Search

by David Monllaó -

Hi Daniel,

Search engines do not need to worry much about the indexing process, search areas are responsible of deciding which info should be indexed, but search engine plugins are fully responsible of all security checkings when retrieving data, if you send a query against all indexed data only checking the access when you already got the results back from the engine you will probably get 0 results for the current user in any site with more than 10 courses (to say a number). More info in https://docs.moodle.org/dev/Search_engines#Retrieve_contents.

We are already receiving proposals for this GSOC project, if you just wanted to share an elasticsearch plugin proof of concept many thanks, but if you want to develop a serious plugin that people can use in their sites I would wait for GSOC outcome, we would not like to waste anyone's time.

As Eric said, the search API and search engine plugintype interfaces will not be final until 3.1 is released, we are still refining it.

In reply to Daniel Neis Araujo

Re: Writing Moodle's Global Search

by Cesar Resendiz -
Hello Daniel, 

I installed elasticserach in my moodle 3.2, I followed the instructions that you have in Github but I did not manage to activate the search.


PasoEstado
1. Seleccionar motor de búsquedaElasticsearch
2. Habilitar áreas de búsqueda
3. Configurar el motor de búsqueda
4. Indexar datosNo
5. Habilitar búsqueda global

When I try to search, I get the following message: The search engine is not available. Please contact your administrator.

Where do you think the problem is?

In reply to Cesar Resendiz

Re: Writing Moodle's Global Search

by Mike Churchward -
Picture of Core developers Picture of Plugin developers Picture of Testers

The message indicates that it cannot find the elasticsearch server you specified in the plugin setup. Do you have an Elasticsearch server running somewhere that you can use (See https://www.elastic.co/)?