Spotlight: Difference between revisions

From SambaWiki
No edit summary
mNo edit summary
 
(23 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Introduction ==
== Introduction ==
This page is meant to give an overview about compiling and configuring Samba with support for OS X [http://en.wikipedia.org/wiki/Spotlight_(software) Spotlight] support.
This page is meant to give an overview about compiling and configuring Samba with support for macOS [http://en.wikipedia.org/wiki/Spotlight_(software) Spotlight] support.


Samba uses [https://wiki.gnome.org/Projects/Tracker Gnome Tracker] as search engine, search tool and metadata storage system.
Samba supports using either [https://www.elastic.co/products/elasticsearch Elasticsearch] or [https://wiki.gnome.org/Projects/Tracker Gnome Tracker] as search engine, search tool and metadata storage system.


See the following pages for details on how to set this up:
== Compiling ==
First, you need the following packages installed: '''tracker''' and '''libtracker-sparql-dev''', the exact name may be different on your system.


* [[Spotlight with Elasticsearch Backend]]
Next, configure samba with --enable-spotlight:
* [[Spotlight with Gnome Tracker Backend]]
<pre>
$ ./configure ... --enable-spotlight ...
...
Checking for tracker-sparql-1.0 : not found
Checking for tracker-sparql-0.16 : yes
Checking for header tracker-sparql.h : yes
Checking for library tracker-sparql-0.16 : yes
Checking for library glib-2.0 : yes
Checking for library gio-2.0 : yes
Checking for library gobject-2.0 : yes
building with Spotlight support
...
</pre>


The recommended backend for any kind of deployment is '''Elasticsearch'''.
== Prerequisites ==

Depending on the OS, it may be necessary to manually install a some additional packages that are needed for the configuration:
* dconf
* dbus-x11

== Setup on Linux ==
This is for Tracker version 1.0 or newer, for older version see below.

We need a small wrapper that will be used in DBUS service files and ensures this environment variable is always set for all tracker binaries:

<pre>
#!/bin/sh
TRACKER_BUS_TYPE=system /usr/libexec/$1
</pre>

Saves this as <pre>/usr/libexec/tracker-wrapper</pre> and don't forget to make it executable.

=== Create a user under which Tracker will run ===

For security reasons it is highly recommended to create a unprivileged user under which Tracker process will run:

<pre>
# useradd -m mdssvc
# passwd mdssvc
</pre>

The user needs a home directory as Tracker stores its database and configuration inside the users home directory.

Tracker will run as this user when crawling and indexing filesystems, so permissions on filesystems must grant read access to this user as desired.

=== dbus Configuration ===

Tracker doesn't provide a dbus config file for running as a system service yet, so here is one: [[Tracker Config File]]. Thanks Martyn!

This file must be saved as <pre>/etc/dbus-1/system.d/org.freedesktop.Tracker1.conf</pre>

=== dbus Service Files ===

Grab Tracker DBUS service files from [[http://www.samba.org/~slow/files/tracker-dbus-services.tgz Tracker DBUS Service Files]] and save them in:
<pre>/usr/share/dbus-1/system-services/</pre>

=== Start Tracker as a special System Service with systemd ===

Grab the systemd unit files for tracker from [[http://www.samba.org/~slow/files/tracker-systemd-services.tgz Tracker Systemd Unit Files]] and save them in:
<pre>/usr/lib/systemd/system/</pre>

Now enable the services:
<pre>
# systemctl enable system-tracker-store
# systemctl enable system-tracker-extract
# systemctl enable system-tracker-miner-fs
</pre>

...and start services:
<pre>
# systemctl start system-tracker-store
# systemctl start system-tracker-extract
# systemctl start system-tracker-miner-fs
</pre>

=== Configuring which Paths to index in Tracker ===
==== GUI ====
If you can do a graphical login, depending on the OS the GUI may offer to configure which paths to index.

Login as the user ''mdssvc'' and eg on Fedora launch '''Settings''' and then goto ''Search''.

==== Commandline ====
Tracker uses Gnome GSettings which mean settings are stored in the user's home directory. Changing settings can be done by running the command '''gsettings''', but this requires a '''user''' dbus session to be active:

<pre>
# su - mdssvc
mdssvc$ dbus-launch --sh-syntax > .dbus_settings
mdssvc $ . .dbus_settings
</pre>

Once you have sourced '''.dbus_settings''' you can run '''gsettings''' command:
<pre>
mdssvc $ . .dbus_settings
mdssvc $ gsettings list-recursively | grep Tracker
org.freedesktop.Tracker.DB journal-chunk-size 50
org.freedesktop.Tracker.DB journal-rotate-destination ''
...
org.freedesktop.Tracker.Extract wait-for-miner-fs false
mdssvc $ gsettings set org.freedesktop.Tracker.Miner.Files index-recursive-directories "['/Volumes/spotlight']"
mdssvc $
</pre>

=== Testing ===

<pre>
mdssvc$ export TRACKER_BUS_TYPE="system"
mdssvc$ /opt/tracker/libexec/tracker-store -v 3
...
</pre>

=== Configuring Samba ===
You can run the Spotlight RPC service '''mdssvc''' either as an embedded service or as separate RPC daemon by enabling the '''mdssd''' RPC daemon:

<pre>
[global]
....
rpc_server:mdssvc = embedded
....
</pre>

or
<pre>
[global]
....
rpc_daemon:mdssd = fork
rpc_server:mdssvc = external
....
</pre>

Finally enable Spotlight searching on a per share basis:
<pre>
[share]
path = /some/path
spotlight = yes
</pre>

=== Starting Samba ===
One must add
<pre>
export TRACKER_BUS_TYPE=system
</pre>
to the Samba init script, or, in case of starting Samba with systemd add
<pre>
[Service]
...
Environment="TRACKER_BUS_TYPE=system"
...
</pre>

to the systemd unit file.

== Setup on FreeBSD ==
This is for Tracker version 1.0 or newer.

We need a small wrapper that will be used in DBUS service files and ensures this environment variable is always set for all tracker binaries:

<pre>
#!/bin/sh
TRACKER_BUS_TYPE=system /usr/libexec/$1
</pre>

Saves this as <pre>/usr/libexec/tracker-wrapper</pre> and don't forget to make it executable.

=== Create a user under which Tracker will run ===

For security reasons it is highly recommended to create a unprivileged user under which Tracker process will run:

<pre>
# useradd -m mdssvc
# passwd mdssvc
</pre>

The user needs a home directory as Tracker stores its database and configuration inside the users home directory.

Tracker will run as this user when crawling and indexing filesystems, so permissions on filesystems must grant read access to this user as desired.

=== dbus Configuration ===

Tracker doesn't provide a dbus config file for running as a system service yet, so here is one: [[Tracker Config File]]. Thanks Martyn!

This file must be saved as <pre>/etc/dbus-1/system.d/org.freedesktop.Tracker1.conf</pre>

=== dbus Service Files ===

Grab Tracker DBUS service files from [[http://www.samba.org/~slow/files/tracker-dbus-services.tgz Tracker DBUS Service Files]] and save them in:
<pre>/usr/share/dbus-1/system-services/</pre>

=== Start Tracker as a special System Service with systemd ===

Grab the systemd unit files for tracker from [[http://www.samba.org/~slow/files/tracker-systemd-services.tgz Tracker Systemd Unit Files]] and save them in:
<pre>/usr/lib/systemd/system/</pre>

Now enable the services:
<pre>
# systemctl enable system-tracker-store
# systemctl enable system-tracker-extract
# systemctl enable system-tracker-miner-fs
</pre>

...and start services:
<pre>
# systemctl start system-tracker-store
# systemctl start system-tracker-extract
# systemctl start system-tracker-miner-fs
</pre>

=== Configuring which Paths to index in Tracker ===
==== GUI ====
If you can do a graphical login, depending on the OS the GUI may offer to configure which paths to index.

Login as the user ''mdssvc'' and eg on Fedora launch '''Settings''' and then goto ''Search''.

==== Commandline ====
Tracker uses Gnome GSettings which mean settings are stored in the user's home directory. Changing settings can be done by running the command '''gsettings''', but this requires a '''user''' dbus session to be active:

<pre>
# su - mdssvc
mdssvc$ dbus-launch --sh-syntax > .dbus_settings
mdssvc $ . .dbus_settings
</pre>

Once you have sourced '''.dbus_settings''' you can run '''gsettings''' command:
<pre>
mdssvc $ . .dbus_settings
mdssvc $ gsettings list-recursively | grep Tracker
org.freedesktop.Tracker.DB journal-chunk-size 50
org.freedesktop.Tracker.DB journal-rotate-destination ''
...
org.freedesktop.Tracker.Extract wait-for-miner-fs false
mdssvc $ gsettings set org.freedesktop.Tracker.Miner.Files index-recursive-directories "['/Volumes/spotlight']"
mdssvc $
</pre>

=== Testing ===

<pre>
mdssvc$ export TRACKER_BUS_TYPE="system"
mdssvc$ /opt/tracker/libexec/tracker-store -v 3
...
</pre>

=== Configuring Samba ===
You can run the Spotlight RPC service '''mdssvc''' either as an embedded service or as separate RPC daemon by enabling the '''mdssd''' RPC daemon:

<pre>
[global]
....
rpc_server:mdssvc = embedded
....
</pre>

or
<pre>
[global]
....
rpc_daemon:mdssd = fork
rpc_server:mdssvc = external
....
</pre>

Finally enable Spotlight searching on a per share basis:
<pre>
[share]
path = /some/path
spotlight = yes
</pre>

=== Starting Samba ===
One must add
<pre>
export TRACKER_BUS_TYPE=system
</pre>
to the Samba init script, or, in case of starting Samba with systemd add
<pre>
[Service]
...
Environment="TRACKER_BUS_TYPE=system"
...
</pre>

to the systemd unit file.

== Tracker versions older than 1.0.0 ==
Tracker uses DBUS for IPC between Tracker processes and processes that want to run queries against the Tracker metadata store. Therefor you have to carefully start Samba, a dbus-daemon instance and Tracker ensuring all processes are then able to communicate with each other over the correct DBUS bus: [[Samba Spotlight Start Script]]. The magic that makes this work is the environment variable DBUS_SESSION_BUS_ADDRESS.

The dbus-daemon that is started from the script need its config file with a matching DBUS session address, here's an example: [[Spotlight DBUS Configuration File]].

Another point to keep in mind is the location where Tracker stores its config and database. Tracker was developed with a per-user use case in mind, as such configuration and data is put in locations relative to the users home directory of the uid of the Tracker processes. By changing the value of the HOME environment variable (a more fine grained approach is modifying the environment variables of the [https://developer.gnome.org/basedir-spec/ XDG basedir specification]).

=== Configuring Tracker and using Tracker tools ===
In order to use the Tracker commandline tools, you have to run a root shell and export the DBUS_SESSION_BUS_ADDRESS variable to the shell environment. The value must match the path from the [[Spotlight DBUS Configuration File]]:
<pre>
export DBUS_SESSION_BUS_ADDRESS="unix:path=/var/run/samba/spotlight.ipc"
</pre>

Now you can tell Tracker which directories to index:
<pre>
# gsettings set org.freedesktop.Tracker.Miner.Files index-recursive-directories "['/Volumes/spotlight']"
</pre>

For older Linux versions where Gnome settings are not yet migrated to gsettings and still use dconf, use:
<pre>
# dconf write /org/freedesktop/tracker/miner/files/index-recursive-directories "['/testshare']"
</pre>

Latest revision as of 09:13, 19 March 2020

Introduction

This page is meant to give an overview about compiling and configuring Samba with support for macOS Spotlight support.

Samba supports using either Elasticsearch or Gnome Tracker as search engine, search tool and metadata storage system.

See the following pages for details on how to set this up:

The recommended backend for any kind of deployment is Elasticsearch.