Share » Forums » Install & configuration » Removing objects takes loooong time,...

Removing objects takes loooong time, can't seem to disable solr indexing for it.

Removing objects takes loooong time, can't seem to disable solr indexing for it.

Tuesday 12 January 2010 9:46:58 am - 7 replies

Modified on Friday 15 January 2010 6:38:27 am by Chris Nelson

Author Message

Robin Muilwijk

Saturday 16 January 2010 1:24:39 am

Hi Chris,

Disabling the re-indexing means that after you purged and imported your products, the solr index file is not consistent any more. So this would require a clean re-index, which in turn would most likely run for some time as well. Is disabling the re-indexing really an option?

You might want to keep an eye on the following thread:
http://share.ez.no/forums/extensions/ez-find-update-index-clean-taking-too-much-server-resources

Board member, eZ Publish Community Project Board - Member of the share.ez.no team - Key values: Openness and Innovation.

LinkedIn: http://nl.linkedin.com/in/robinmuilwijk // Twitter: http://twitter.com/i_robin // Skype: robin.muilwijk

Chris Nelson

Saturday 16 January 2010 7:28:42 am

Thanks for the reply!

The re-indexing is very fast, actually, when done after-the-fact. For example, DelayedIndexing is enabled, and that prevents indexing during the import, and I do a re-indexing pass after that which is very speedy (just a few minutes). It's only the removal that is still slow (2-3 seconds per object removed!), and Java resources spike during the object removal, so I'm convinced that Solr is doing work on the backend, even though DelayedIndexing is enabled and OptimizeOnCommit is disabled.

So, delaying re-indexing is not a problem...

x

Robin Muilwijk

Saturday 16 January 2010 11:02:44 am

Would it be worth a try to completely de-activate eZ Find? See http://ez.no/doc/extensions/ez_find/2_1/installation, same way as you activate the extension. Of course just before you purge and re-import all of your products?

I have no idea on the impact of disabling eZ Find, but if you do a re-indexing anyway, I don't think it's going to harm your setup.

-- Robin

Board member, eZ Publish Community Project Board - Member of the share.ez.no team - Key values: Openness and Innovation.

LinkedIn: http://nl.linkedin.com/in/robinmuilwijk // Twitter: http://twitter.com/i_robin // Skype: robin.muilwijk

Paul Borgermans

Saturday 16 January 2010 12:47:14 pm

Removing those objects through the API calls a delete and commit with every object to the backend index. This can be indeed be quite slow when there are many objects.

Can you file an issue for this? In the current trunk, there are optimisations for high write traffic sites, I'll add a config option to eZ Find that disables commits on delete and add a small script that you can use after the delete operations are finished to issue a commit.

(warning: before the commit, deleted objects will still show up in the search and trigger fatal errors as the eZP side calls will fail obviously)

@Robin: disabling ezfind alltogether during this delete will indeed speed up the whole operation as well

An alternative would be to patch Solr, so it also accepts a "commit within" parameter like it does for updates/additions ... I'll have a look at that too, the Java code to accomplish this is quite simple at first sight.

hth

Paul

eZ Publish, eZ Find, Solr expert consulting and training
http://twitter.com/paulborgermans

Robin Muilwijk

Sunday 17 January 2010 12:01:11 am

@Paul, thanks :)

@Chris, you can file an issue here: http://issues.ez.no/HomePage.php? . Tip; when filing the issue, link back to this forum thread.

Thanks, Robin

Board member, eZ Publish Community Project Board - Member of the share.ez.no team - Key values: Openness and Innovation.

LinkedIn: http://nl.linkedin.com/in/robinmuilwijk // Twitter: http://twitter.com/i_robin // Skype: robin.muilwijk

Chris Nelson

Monday 18 January 2010 6:18:56 am

@Paul, thanks :)

@Chris, you can file an issue here: http://issues.ez.no/HomePage.php? . Tip; when filing the issue, link back to this forum thread.

Thanks, Robin

Thank you for your responses. I will file the issue today, after I'm out of meetings.

x

Gabriele Francescotto

Wednesday 24 February 2010 3:36:09 am

Dear Paul,

do you have any news about the script or the patch you mentioned above?

I have too the problem described by Chris, using ezfind 2.1 on ez 4.2

Many thanks,

Gabriele

OpenContent [free software solutions]
via Verdi 19, 38100 Trento (TN) Italy
www.opencontent.it
skype : gabricocek1
twitter: gabricocek

You must be logged in to post messages in this topic!

36 542 Users on board!

Forums menu