2015-08-20 Carlos Garnacho Release 1.5.2 2015-08-19 Carlos Garnacho libtracker-miner: Mark nodes being removed We don't want to recurse on these, this turns the last tracker_indexing_tree_remove() call a no-op if happens to be called recursively within a ::directory-removed signal handler. Revert "libtracker-miner: Postpone TrackerIndexingTree::directory-removed emission" This reverts commit 5b4ab760c4782d1b7a098605b69d547c9466d20e. There's code relying on poking the removed directory flags one last time during ::directory-remove, so we must prevent from recursion in some other ways. Spotted during make distcheck. libtracker-miner: Check the crawler cancellable before destroying it It won't be set unless tracker_crawler_start() was called, so there's a chance it could be NULL on finalize. Spotted on make distcheck tests: Fix make check We now return a GError in the only tested condition, so look for this at least tracker-miner-fs: Keep cache of IndexFile requesters on directories The senders for all DBus requests to the IndexFile method on directories will be now watched, the dbus presence of the senders will control the lifetime of the directory on the indexed directories tree. There may be multiple requests on a same directory, in such case the directory will be indexed/monitored for as long as there is alive requesters on it. Requests on already indexed directories (or children of recursively indexed ones) will be silently ignored. Unmounts will also silently drop the IndexFile listeners, applications should issue new requests on volume mounts if desired. The patch is loosely based on initial work from Felipe Borges. https://bugzilla.gnome.org/show_bug.cgi?id=680834 2015-08-18 Sam Thursfield libtracker-miner: Deprecate tracker_miner_fs_add_directory_without_parent This has been unused since commit 74d20884c870c2f54d93d3c01824b0994629c5d3. 2015-08-17 Jordi Mas Update Catalan translation 2015-08-15 Carlos Garnacho libtracker-miner: Invalidate TrackerDecoratorFS cache on (pre)unmount This ensures tracker-extract promptly drops all its current items when volumes are being unmounted, otherwise the already cached items will keep being processed, possibly preventing unmount from happening. libtracker-miner: Postpone TrackerIndexingTree::directory-removed emission Emitting before actually detaching the file from the GNode tree won't be safe if the signal results in reentrancy on tracker_indexing_tree_remove() libtracker-miner: Drop cache on TrackerDecorator pause/resume The cached data might be old by the time the miner is resumed, so just fetch some data from scratch. libtracker-miner: Ensure TrackerFileNotifier stops on crawler interruption finish_current_directory() shouldn't jump to the next directory in the current root if crawling is being interrupted there. Now that we unset priv->current_index_root as a result of interruption, we can avoid doing that ad-hoc on indexing_tree_directory_removed(). libtracker-miner: Fix TrackerCrawler cancellation On tracker_crawler_stop()/cancellation, the various async callbacks would poke memory that is no longer valid (The DataProviderData and DirectoryRootInfo structs will be immediately destroyed, before the callbacks are run). This fixes multiple crash scenarios upon cancellation, the crawler has been also simplified to use a single cancellable, since we'll be just running one data provider/enumerator at a time. tracker-extract: Plug error leak libtracker-extract: Plug error leak libtracker-extract: Plug leak The string would be leaked if we end up with too low confidence from libicu. 2015-08-11 Sam Thursfield functional-tests: Improve 'manual start' mode This mode now allows you to run the tests against already-running Tracker processes. A lot of tests depend on specific miner-fs configuration so they might not work, but it can be useful. You can still start the test and then manually start the Tracker process if you want. Note that enabling manual start on the commandline probably doesn't work (commandline options for these tests are pretty broken) so just change the options.is_manual_start() function to 'return True' if you want to use this mode. functional-tests: Clean up miner tests a bit Update .gitignore 2015-08-09 Necdet Yücel Updated Turkish translation 2015-08-08 Felipe Braga Updated Brazilian Portuguese translation 2015-08-04 Carlos Garnacho tracker-extract: Remove SVG handling code from the gstreamer module This is now unused. tracker-extract: Use dummy fastpath for svg extraction The gstreamer based extraction is less than useful for our purposes, it could spend considerable amounts of cpu time and memory in building a pixel representation of the image (which is useless for us), and would extract nothing relevant from the file (w/h are meaningless on scalable images, other details such as geolocation just don't apply here). So SVG through gstreamer was a very expensive no-op, it can be safely made to use dummy extraction, and it will still be set the corresponding rdf:types. This is our side of the fix for: https://bugzilla.gnome.org/show_bug.cgi?id=748608 libtracker-extract: Add builtin dummy extractor For rules with no ModulePath key (i.e. those we only want to set rdf:types for) we relied on the lack of generic extractors that might handle the mimetype. If there was one such generic extractor, the TrackerMimetypeInfo iterator would skip the empty rule and directly attempt the generic extractor. Because it is conceivable that no-op is desired for some files, we shouldn't attempt to skip it, nor run further modules after the dummy rules, so just give this dummy fastpath a presence that tracker-extract will make sense of. tracker-extract: items==0 doesn't imply failure There may be files where there's just nothing to be extracted, make these fall through the success code paths. If a module fails for any reason, it must return FALSE on the get_metadata func. tracker-extract: Handle correctly empty insertions There may be files for which is valid to say "there's nothing to extract here" without resorting to error paths, in that case we must insert the dataSource, but skip the empty statements for the file. tracker-extract: Handle NULL modules This will be necessary when we have an internal dummy function. 2015-08-03 Andres Gomez libtracker-common: String to date conversion to return with GError when null string Previously we were failing with a GLIB CRITICAL. Now, return gentlier and with a filled GError. https://bugzilla.gnome.org/show_bug.cgi?id=752544 2015-08-03 Pedro Albuquerque Updated Portuguese translation 2015-08-02 Aurimas Černius Updated Lithuanian translation 2015-07-31 Piotr Drąg Updated Polish translation 2015-07-31 Martyn Russell build: Fix bash completion dir problem with make install, distcheck - Support JHBuild too - Allow disabling altogether - Fix DISTCHECK flags Conflicts: Makefile.am 2015-07-31 Ting-Wei Lan build: Fix return value error for tracker_file_notifier_get_file_type https://bugzilla.gnome.org/show_bug.cgi?id=752900 2015-07-31 Martyn Russell tracker: Fixed search with -f or --files , broken expression https://bugzilla.gnome.org/show_bug.cgi?id=752871 2015-07-27 Dušan Kazik Updated Slovak translation 2015-07-26 Carlos Garnacho rss: cater for nmo:communicationChannel cardinality changes Now that mfo:FeedMessages can reference multiple mfo:FeedChannels, ensure we set link messages with all channels we receive these from. ontology: Remove cardinality limits on nmo:communicationChannel For at least the case of feed messages (the only user of nmo:communicationChannel inside Tracker), this does not quite represent reality, a same message can come from different channels (eg. same developers across several planet feeds), so remove the cardinality limit. The change also affects the indices created on the nmo:Message table, there will be an implicit index for the multivalued field, so there's no need for tracker:indexed, the secondary index shared with nmo:receivedDate has changed into making an index on nmo:receivedDate itself, it should yield similar results together with the index on the new standalone table for nmo:communicationChannel. Regarding compliance with Nepomuk, nmo:communicationChannel got stuck forever as an enhancement request at http://dev.nepomuk.semanticdesktop.org/ticket/722, so it's not something to care about here. libtracker-data: Cope for cardinality changes overwriting indices There's otherwise name collision between the old single-valued index and the new multivalued table. The index for the multivalued table will be created afterwards. rss: Unify website handling We now keep per feed item a hashtable of websites, so we can filter duplicates and point to the right URN/IRI in the insert queries. rss: Perform extraction/insertion of feed items at once The async ASK/INSERT pattern per-feed was vulnerable to nie:url collisions if we happened to process several feed items pointing to the same URL simultaneously. Change this so we first query the older items, and then we check items are really up to date, inserting only the newest of occurrences, and updating the database if the content changed for real (eg. the post was updated). This is done in a single update_array operation for all feed items in a channel. rss: Refactor mfo:FeedMessage sparql insert creation into a separate function No functional changes here. 2015-07-25 Carlos Garnacho rss: Optimize deletes We now just do 2 queries, one that deletes nmo:communicationChannel from all related mfo:FeedMessages when a mfo:FeedChannel is deleted, and another that garbage collects all mfo:FeedMessages that have no nmo:communicationChannel. This approach is compatible with any possible cardinality changes to nmo:communicationChannel. libtracker-data: Clean up stale URIs on startup The Resource table can be left populated with stale resources, in order to avoid these from accumulating, clean these up during startup. This assumes all miners and clients will be educated and will clean up all references to deleted resources (triples of the form {?urn some:prop ?deleted}) at the moment deletes happen, and always during the lifetime of the previous run of tracker-store. If there's sloppy miners/clients that don't fully unlink the resource, or users mess with the lifetime of tracker-store in the middle of such maintenance operations, we will leave such stale values around the database, possibly confusing queries where the predicate is used in the WHERE section. Short of having fully refcounted resources, seems like a minor drawback. Revert "libtracker-data: delete elements from the Resource table" This reverts commit ae3ab553d0e5faf0dadc85586dac0c0ee4443394. This approach is a bit too drastic, as it may confuse queries triggered by the deletion of the resource, the SQL translation of DELETE queries using the deleted resource in the WHERE will fail because there is no matching URI for that row ID anymore. Another less drastic approach is done in the next commit. 2015-07-24 Carlos Garnacho rss: Keep the channels by tracker:id() in the hashtable We won't be able to fetch the old URNs for the delete mfo:FeedChannels at the time of receiving the GraphUpdated notifications, we can only store these by tracker:id() as that's the only thing we readily have. rss: Simplify GrssFeedChannel list creation Simply fetch the hashtable values, instead of overcomplicating the old/new checks. 2015-07-24 Marek Černocký Updated Czech translation 2015-07-24 Carlos Garnacho rss: fix typo Destroy, not free. rss: Set website url as a nfo:WebSite nco:websiteUrl has to point to a rdfs:Resource according to the ontology, so make the string a full blown nfo:WebSite, with its nie:url. 2015-07-22 Carlos Garnacho libtracker-miner: Fallback to basename checks on hidden files When we receive DELETED events on files, the TrackerFileNotifier/ TrackerIndexingTree attempt to find out if its hidden and should thus be ignored. However the g_file_query_info() evidently fails at that time, with the file already deleted, so we think the file is not hidden and the delete operation gets propagated to the TrackerMineFS. Fix this by resorting to basename checks as a last resort, so we do the right thing even after the file is gone. 2015-07-22 David King libtracker-data: Fix printf string format Introduced by commit 81135ae19c727df716d0c0945869462e75a53497. https://bugzilla.gnome.org/show_bug.cgi?id=752717