Official Search Module

almostcompletely

I seem to have installed the official search module correctly (Manual Install). The Tool menu item appears in Admin along with a list of my pages. If I click "reindex", nothing happens. The official screenshot shows that I should expect a list of indexed items - I get nothing. Checking the database table, it's empty.

I've obviously missed a vital configuration item. Anyone got an idea what?

TIA...

admin

I would first copy the fuel/modules/search/config/search.php file and add it to your fuel/application/config/search.php. This allows you to overwrite parts of the configuration and not worry about messing with the code in the search module. Then, in that file, I'd check the "delimiters" configuration and change the "'<div id="main">'" delimiter to be something that matches your HTML code structure. The delimiter tells the search engine which node(s) to use in your site for indexing.

almostcompletely

Got it. Now the whole module makes sense (maybe the docs could do with an extra line or two to point this out - for the thickies like me?).

I'm still having trouble indexing though. I've got some 4000+ pages to do. If I select a few from the reindex Admin page, they get indexed however if I elect to index all the pages, it times out:

"Fatal error: Maximum execution time of 300 seconds exceeded in /var/www/49squadron/fuel/modules/fuel/libraries/Validator.php on line 431"

On my production system, I can edit the PHP.ini settings to avoid this but on the public (shared hosting) site, this is not possible. Any suggestions?

It doesn't seem possible to select a few pages at a time for reindexing - each time the database table is truncated and it starts again from scratch....

I "could" build the index on my production system and copy the table across but I need to enable this for the site administrator to do as they add new content.

Cheers.

admin

Unfortunately, there may not be an easy way if you can't make changes to the PHP.ini settings to allow for more memory and will probably require some means of copying the data over to your production environment.

Selecting a few pages from the admin shouldn't be truncating the table (only updating). Selecting Reindex without selecting any pages will truncate it though. You can also set it up to use a tmp table while indexing in case of any errors. There is a config parameter of "user_temp_table" (which I just posted a fix to be "use_temp_table" that you can set in the config to TRUE).

It's possible for you to index a remote site locally by setting the "base_url" property in the fuel/modules/search/config/search.php config (I'd recommend copying this file to fuel/application/config/search.php to allow you to make project specific configuration changes). However, you would still have to find a separate way to copy over the fuel_search table to your production server at this time (that would be a good feature to copy it over to a different server though).

Also, there is a hook you can setup where it will incrementally index pages in the CMS and any simple module that has a preview_path set. To add this, add the following to your fuel/application/hooks/hooks.php file (you may need to periodically do a full update to the index):
include(SEARCH_PATH.'config/search_hooks.php');

Lastly, as an FYI, that's a lot of pages and may take a long time to index. It may not be a good idea to run that on the same server anyway.

almostcompletely

Ha! thanks for that fix. I was wondering why it kept using the tmp table when my config had it switched off.

I now have it incrementally adding to the index if I select a few pages from the list at a time. I think this will be acceptable to the administrator...

I'm a little confused about my extra simple modules though. If I reindex without selecting pages, pages published by my simple module are indexed. They are not if I select items from the list - and my module isn't listed so I can't choose it...

admin

Did you specify a preview_path for you module in the MY_fuel_modules.php file? That is required in order for there to be URLs generated for your module.

Also, if I was thinking there may be something that can be done with the Sync module and the cron module to sync the database fuel_search table (there's a "db_sync_prefs" preference in which you can tell it which tables to sync).

almostcompletely

Interestingly, I didn't have a preview_path set. But indexing all records includes the module's URLs (because the links were within existing pages?)

I've now added a preview_path and I have a list of module pages - excellent - this will allow the administrator to manually & incrementally build the index.

For the record (having read up on preview_path in the docs), I originally set it to
'preview_path' => 'personnel_index/detail/{slug}'
...that being the URL path to my records via the controller. It turns out it just needed my module name
'preview_path' => 'personnel'

almostcompletely

This is doing my head in

My development server has this working perfectly, having duplicated the configuration to my production server, I still get no modules listed.

Both systems are v1.06

./application/config/search.php
$config['search']['index_modules'] = TRUE;

./application/config/hooks.php
include(SEARCH_PATH.'config/search_hooks.php');(though with this missing it still worked on development server)

./applications/config/MY_fuel_modules.php
$config['modules']['personnel'] = array('module_name' => 'Personnel', 'display_field' => 'reference', 'instructions' => '', 'sanitize_input' => array('template','php'), 'preview_path' => 'personnel');

On the live server, the output from ./fuel/modules/search/controllers/search_module.php (line 41)
$fields['pages'] = array('type' => 'multi', 'options' => $this->fuel->pages->options_list('all', TRUE, FALSE));

...gives me an array consisting of the cms, modules and views. On the development server, the modules section contains module stuff. On the live server, the modules section is a replica of the cms section.

On the live server, I've edited ./application/config/environments.php so Fuel ran in both development and then production mode with no change.

I'm at a loss as to why my live system refuses to acknowledge my module. Anyone got any suggestions?

admin

Does the fuel_search table have different data between the environments? The search module has a search sub module (simple module) in it that has a preview_path value of 'location' => '{location}' and therefore gets included in that list of all pages.

almostcompletely

Now, by truncating my fuel_search table on development, the personnel module content is missing from the reindex selector... Setting the preview_path seems to have no effect on populating this list. I can't think of anything specific I did on populating the table between the servers - basically selected a few items, indexed, then did a few more. At the end of indexing development and then adding the preview_path the module (ie., it's pages) suddenly appeared in the selector - the same process on live failed to produce personnel content in the selector.

Populating the selector seems to happen via $this->fuel->pages->options_list('all', TRUE, FALSE)) in the search_module.php controller and in turn picks up the modules portion from $this->modules in the Fuel_pages library(line #151). How does that get populated?

almostcompletely

>> The search module has a search sub module (simple module) in it that has a preview_path value of 'location' => '{location}'

Can't seem to find that in any of the search module code

admin

If you truncated the table and it affected your list, then that tells me that your dropdown list is using the data from the search index to help create that page list (by using the preview_path found here fuel/modules/search/config/search_fuel_modules.php).

Do you have any CMS pages that are showing up in that list and are they also on production? Have you checked out the source code for that method that generates the pages fuel/modules/fuel/libraries/Fuel_pages::options_list()?

almostcompletely

I think you've confirmed what I've been suspicious of for some time - the search module cannot explicitly index the contents of tables populated by home-rolled simple modules. It seems that if module content happens to be referenced elsewhere on a normal CMS page, then the chances are it might get picked up if you have preview_page set.

Trouble is, 9/10 times, my personnel module is the sole location for the data to be indexed - in this case, it will never be indexed.

If you agree, and I'm not deluded in some fashion, I guess I'll need to work on the search code...

admin

So are you saying that the module data you are wanting to index doesn't exist on a page somewhere on your site?

almostcompletely

In some cases, yes. The data is exposed on the result of a search. There's not a page with massive list of links to personnel records - perhaps I should have one/

almostcompletely

Also, the module records are only exposed through the module. eg ./personnel_index/detail/J_DOE. A list of people is available via ./personnel_index/mylist. But I can't add the module to the search box...

mylist exists on the main menu but not the page content specified by "delimiters" config item.

almostcompletely

So I've now created a module view "personnel_index/all". It's not referenced on the site itself but you can navigate to it directly. This page lists all "personnel_index/detail/XXX" links.

By manually adding "personnel_index/all" to the search list view in Admin, it's now available in the "Reindex" list.

By my understanding, with "preview_path" set up, if I reindex this page, I should get a full list of detail records in the index? It doesn't happen

admin

It would become available in that reindex list because the search module itself has the "preview_path" => "{location}", set and since you created a new record with a location of "personnel_index/all", it will appear in that list. However, if you select that page to index, it will only index that one page and not continue crawling to the other pages listed on that page. Also, reindexing the entire site would wipe out that record and not append it since it's a hidden link and not navigable.

I think for this to be indexed, it needs to be reachable from some page on the site so it can be crawled.

almostcompletely

I think I've cracked it. Fingers crossed. Thanks for your input.