Install tracker by default in 18.04 LTS?

[jbicha]

The Ubuntu Desktop Team is trying to decide whether to install tracker by default in Ubuntu 18.04 LTS. Tracker is a file indexing service and provides several benefits if installed. On the other hand, there are reports of tracker sometimes causing high CPU or other performance issues.

Therefore, this is a call for testing. It would be a big help if anyone can provide specific data about the performance impact of installing and enabling tracker.

How to Install

Run this command in a terminal in either Ubuntu 17.10 or 18.04 then log out and log back in:

sudo apt install tracker

Benefits

1. It speeds up searching for files in the Files app.

2. It enables full-text search in the Files app. In other words, you can look for files that contain specific words, instead of searching just by filename.

3. It allows the Batch Rename feature in the Files app able to rename based on file metadata. For instance, you can use the Artist name for properly tagged music files as part of the new filename.

4. It enables file and folder search in the Activities Overview on GNOME.

Other Info

You can customize the folders it indexes from Settings > Search > gear button.

Tracker is required by some GNOME apps that aren’t installed by default in Ubuntu 17.10 like Music and Photos so it might already be installed on your computer.

[nickthom]

I remember when I had to disable tracker on every distribution I had GNOME installed (Arch Linux, openSUSE, Fedora) because of the impacts you describe. I don’t know if things got better since then. I’ll give it a try.

[oSoMoN]

Just installed tracker. After restarting my session, the initial metadata extraction is visibly taxing my CPU:

Memory usage during extraction is reasonable so far (tracker-miner-fs < 150MB, tracker-extract < 400MB), considering I’m not doing any memory-intensive computation otherwise.

One interesting thing: the initial indexing and metadata extraction appears to be complete now, tracker-miner-fs and tracker-extract are idle, but they haven’t freed the memory they used. Is that expected?

[khurshid-alam]

Tracker imposes heavy cost on battery life and hard disk due to it’s intensive nature of indexing everything and thus constantly writing data to disk. You can check this with io-top. Also gvfs-metadata suddenly started using high cpu after I install tracker.

On battery with tracker (default configuration), my laptop (Dell XPS) only lasts 4 hour on moderate usage, without tracker it lasts more than 8 hours.

More from the bug comments:

First time I ran tracker never stopped indexing. I monitored tracker-miner-fs.log and also with (/usr/libexec/tracker-extract -v 2). At some point you would expect logging to stop when it becomes idle. But it didn’t. Only after tweaking and hard resetting situation improved dramatically.

I feel the default configuration is very bad. I changed following things and after that the situation is greatly improved.

-- enable-writeback false
-- index-optical-discs false
-- index-on-battery false
-- index-removable-devices false
-- ignored-directories ['core-dumps', 'CVS', 'lost+found', 'po', 'vendor']
-- ignore-stop-words true
-- ignore-numbers true
-- max-words-to-index 1000
-- removable-days-threshold 3 ? (I don't not understand this)

But now there won’t be any gui to configure this. Gnome is deprecating tracker-preferences. So we only have dconf-editor to configure.

[nickthom]

Maybe tracker behavior depends on other things too.

I have installed it and no negative impact so far.
BUT, I have an SSD, the installation is fresh thus not many files and folders to index.

tracker status
Currently indexed: 20 files, 10 folders
Remaining space on database partition: 87,3 GB (89,04%)
All data miners are idle, indexing complete

However, my syslog flood with messages about tracker-store

Maybe relative to this bug ?

[fossfreedom]

From Ubuntu Budgie POV - we shipped with tracker in 17.04.

Anecdotal evidence from users visiting our community room over the 6 month period (about a dozen or so) commented on the following:
long processes running/fan’s blowing/high cpu and memory usage increasing (100Mb to 200Mb).

This was tracked down to tracker. Removing it resolved it for 17.04. We didn’t ship with tracker for our 17.10 release. So far, no-one has commented as to inability to search for stuff via nautilus.

And yes - the team have urged people to file bugs - but sure as eggs is eggs … very few have

sorry I havent got anything more concrete to share.

[bashfulrobot]

Just to add to @fossfreedom comment. We had even gone as far as removing any apps that required tracker (in 17.10). It dropped our resource usage, and support issues (noticeably). It would be nice that if Ubuntu proper was looking to use it that maybe they include it as part of their seed vs. adding it in such a way that all the flavours automatically inherit it. For clarification - would this just apply to Ubuntu proper? Just want to be sure I understand the context here.

[stephen-d-allen]

For me it used to back in early gnome versions - lately not bad at all. I have it on low priority. I would be hampered without its functionality as on large disks its the only way to search for things fast. Personally many users here use the search function in the dash. As designed it’s a great feature.

The people that are having problems, must shut down their computer - Our workstations stay up 24/7 so indexing is done when not being used, and like I said as a low priority process.

I remember other OS users saying the same about their Searchlight etc. The benefits far outweigh the “issues” some are experiencing in my opinion.

[jbicha]

@bashfulrobot

For clarification - would this just apply to Ubuntu proper?

Yes, the Ubuntu Desktop team is only making this decision for the flagship Ubuntu Desktop. Ubuntu flavors like Ubuntu Budgie are free to continue making their own decisions about whether to install or not install tracker by default. I appreciate the feedback from Ubuntu Budgie’s experience.

[GrindamN]

As a person who tested the tracker in Budgie, it really made the impact when it was removed. Whether it was the desktop with low memory or laptop. It was really noticeable especially on the hard drives where there would be constant writing activity. Removing the service alongside its dependent applications, allowed us to get snappier distro with lesser memory usage, better battery, and less hard disk activity. We haven’t noticed as @fossfreedom and @bashfulrobot said problems regarding resource usage or other issues that users needed support with after we removed tracker which correlated with less support requests.

[didrocks]

Doing regular backups with deja-dup (duplicity as the backend), I notice that tracker is taking a lot of core and CPU everytime a backup is performed (and you can see the disk is hammered), even if it’s supposively only read operations on that side.

[stephen-d-allen]

That’s not my experience. My personal desktop, does backups daily using deja-dup (Gnome’s backup utility). My 8 cores never get more than 30% during, and haven’t noticed any disk I|O usage of any significance. The only settings that aren’t default on my box is Tracker being set to ‘low-priority’ and I don’t have the destination drive for backups enabled for indexing via the search settings in Gnome-Control-Panel.

[jyaku]

Well, all the Ubuntu 17.4 and Budgie 17.4 anectodes are certainly entertaining to read, but there was a big overhaul in between then and now with the jump from Tracker1 to Tracker2. Chances are that Tracker 2 (Ubuntu 17.10.) is much lighter on resources than old Tracker 1. So old Tracker1 experiences don’t seem too worthwhile.

And CPU cycles and RAM are meant to be used (unless in a power saving mode). I think that it is more important that Tracker goes out of the way easily if the resources are needed by other tasks, indexing just when being eg. idle.

When aiming to save RAM and cycles I would rather try to stop GDM3 sitting in the background for little gain. Collecting data IS worthwhile, for faster searches, for faster program start ups and so on.

And more and more GNOME features start to use tracker, so it will get harder and harder to avoid installing it. I think it would be more future-proof to help sorting out the problems instead investing resources to avoid it (eg. patching out features that depend on it).

[laney]

I guess I better add my +1 vote in here then.

I don’t even remember installing tracker - maybe I got it via a dependency from something else - but I’ve had it on my system for ages. If I notice something behaving badly on my system, it’s usually Firefox - never in my memory have I felt like Tracker was responsible for any bad behaviour.

Yesterday I needed to search in my ~/Documents for a PDF that I knew mentioned the word “treasurer”, but I couldn’t remember the filename of. Thanks to having Tracker installed, I was able to type “treasurer” into Nautilus and it found the file instantly. That was really great.

smem’s “PSS” pins 7610+12256+13393+16152 = 49411K on all tracker processes on my system ATM.

So my Laneyish vote is to go for it and get some targetted and actionable bug reports that we can work on, like if there is a memory leak that is affecting initial indexing, or if something weird happens in the backup usecase.

[bashfulrobot]

@jbicha thanks a ton for the clarification. I appreciate it. Hopefully our little blurb was helpful.

[bashfulrobot]

That’s interesting reading about tracker 1 vs 2. May do some additional testing. The larger take away for us (u budgie) was a drop in support issues. With that being said, will likely play around.

But also since Jeremy mentioned that this doesn’t cause the flavors to go in one direction or the other, or vote/feedback might be kind of moot.

@fossfreedom / @GrindamN - just for visibility.

[switches]

Just as a user I think this should be an option for the install, not a “default” on thing.

I’ve tested tracker a few times (and I’m currently running it on 17.10) the only time I noticed any “lag” or problems with it are connecting/disconnecting drives (external USB has been the worst for me) although I think that could be fixed in the config (I’m just testing it as the stock at the mo, haven’t changed anything… yet).

Other than that it seems to eat CPU time and MEM on a fresh install, but after about two or so hours (depends on the number of files it has to index obviously) it drops right down and basically remains unnoticable (so far atleast).

Only been running it for around a week or so, so it’s not really been hammered yet.

[KristijanZic]

I don’t know if this is related to tracker but nautilus freezes for me when I try to copy anything from nautilus, not just files but also file names while renaming. On my desktop tracker can get out of control and eat my whole ram and crash the system.

[Daedra]

Gnome tracker is dreadful, and I mean DREADFUL, I installed it in 17.10 and it is eating CPU and RAM like crazy and it is far less effective than search in Unity dash or KDE with Baloo. The fact you cannot turn off indexing file content probably contributes a lot to this, I would like it preinstalled, but only if there is a toggle in System Settings where we can disable indexing file contents, KDE’s Baloo indexes hundreds of thousands of files on my partitions VERY quickly when I turn off file content indexing (which usually does not serve much purpose anyway for me). Unity dash search is sorely missed in this new desktop, but tracker has poor performance with its file content indexing turned on by default, I havent been able to find any configuration files where I can turn it off and even if they existed users should be given the GUI option of disabling indexing file contents in System Settings. I believe turning on indexing (without file contents indexing) by default on all partitions mounted in fstab would be the best solution that would mimic the Unity dash the most.

[verretpa]

I like tracker a lot, I’m using it several years already.

But regurlarly, about 3 times a week, I experience serious performance issues, mainly IO-bound. My system is completely unresponsive at those moments.

The reindexing should be performed nightly, but it starts at the most unpredicable times.