WSP

WSP (Windows Search Protocol) support in samba

support in samba

• since samba-4.20 samba ships a command line client for searching using the WSP protocol. The 'wspsearch' cli client does not work against a samba server as it currently does not implement the WSP protocol

WSP server support

• The WSP protocol is not supported upstream in samba yet. However, there are a couple of upstream merge requests currently open

1. Support rawpipe services (servers using named pipes but not using the dcerpc protocol) allowing them to be managed in the same way as dcerpc servers are. See here

2. Allow mapping between authenticated samba user and elastic/opensearch basic user. The allows the samba server (spotlight or WSP) authenticate over http with a basic elastic/opensearch user. See here

3 A merge request with the WSP stand alone server code.

can I try it out

Yes you can, if you are willing you can build from a git branch

  git clone git://git.samba.org/npower/samba.git samba-wsp
  cd samba-wsp
  git checkout -b current_wsp_421_wip origin/current_wsp_421_wip
  ./configure.developer # (and install all the dependencies)
  make install
   

WSP running and testing using elasticsearch

install elasticsearch

using elasticsearch-8.15.2 (latest version at time of writing)

rpm -ivh elasticsearch-8.15.2-x86_64.rpm

take note of the generated built-in superuser 'elastic' (output as part of the rpm install)

if desired change the generated superuser password

/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic

start it

systemctl daemon-reload
systemctl start elasticsearch.service

check if is running

systemctrl status elasticsearch.service

check communication

 curl -k -uelastic:elastic https://127.0.0.1:9200

should respond with

{
 "name" : "localhost.localdomain",
 "cluster_name" : "elasticsearch",
 "cluster_uuid" : "n-AXwOQeTOSddb_p3UXsUQ",
 "version" : {
   "number" : "8.15.2",
   "build_flavor" : "default",
   "build_type" : "rpm",
   "build_hash" : "98adf7bf6bb69b66ab95b761c9e5aadb0bb059a3",
   "build_date" : "2024-09-19T10:06:03.564235954Z",
   "build_snapshot" : false,
   "lucene_version" : "9.11.1",
   "minimum_wire_compatibility_version" : "7.17.0",
   "minimum_index_compatibility_version" : "7.0.0"
 },
 "tagline" : "You Know, for Search"
}

configure elasticsearch

Note: This is a developer setup, not suitable for production, please refer to the elasticsearch documentation for specific information about securing elasticsearch

• disable ssl

for testing it is convenient to be able to easily see the communication between samba and elasticsearch unencrypted, of course ssl can be re-enabled after a working setup has been established.

in /etc/elasticsearch/elasticsearch.yml:

xpack.security.http.ssl:

-  enabled: true
+  enabled: false

Use fscrawler to index files

using latest fscrawler version 2.10 (at time of writing)

• identify (or create) some locations on the filesystem (which are accessible from samba shares) that have content you would like to index
• install fscrawler

unzip fscrawler-distribution-2.10-20240702.144319-374.zip

• create a user to use to communicate with opensearch to populate the index

Here we will use the 'elastic' superuser (just for testing) that comes already setup with elasticsearch. Note: the elasticsearch 'elastic' user is a super user. You might want to consider creating a specific user for fscrawler (or generate an apikey) to use that has appropriate roles assigned. e.g. with 'just enough' privileges to access the index(s) you want to create/modify. (see creating users, creating roles, creating roles, creating API key and associated documentation)

• use fscrawler to create an index

 ./fscrawler-distribution-2.10-SNAPSHOT/bin/fscrawler index_name

you will be prompted (if this is the first time to run the command)

INFO  [f.console] job [index_name] does not exist
INFO  [f.console] Do you want to create it (Y/N)?
answer 'Y'

• edit the config file ~/.fscrawler/index_name/_settings.yaml created in the last step

• configure path fscrawler to index

url: "/path/to/index"

• setup fscrawler to disable ssl when communicating with opensearch

- url: "https://127.0.0.1:9200"
- url: "http://127.0.0.1:9200"

-  ssl_verification: true
+  ssl_verification: false

• don't stop on error (otherwise any problem indexing a specific file will stop the indexing process)

-  continue_on_error: false
+  continue_on_error: true

• setup optional stuff

-  attributes_support: false
-  raw_metadata: false
+  attributes_support: true
+  raw_metadata: true

• run fscrawler again

./fscrawler-distribution-2.10-SNAPSHOT/bin/fscrawler index_name --username elastic --loop 1

WSP running and testing using opensearch

install opensearch

using opensearch-2.15.0 (latest version at time of writing)

OPENSEARCH_INITIAL_ADMIN_PASSWORD=1234?Changeme rpm -ivh opensearch-2.15.0-linux-x64.rpm

start it

systemctl daemon-reload
systemctl start opensearch.service

check if is running

systemctrl status opensearch.service

check communication

 curl -k -uadmin:1234?Changeme https://127.0.0.1:9200

should respond with

{
 "name" : "localhost.localdomain",
 "cluster_name" : "opensearch",
 "cluster_uuid" : "6fJA5WMmSiK2wc4rHdkVvw",
 "version" : {
   "number" : "7.10.2",
   "build_type" : "rpm",
   "build_hash" : "61dbcd0795c9bfe9b81e5762175414bc38bbcadf",
   "build_date" : "2024-06-20T03:27:31.591886152Z",
   "build_snapshot" : false,
   "lucene_version" : "9.10.0",
   "minimum_wire_compatibility_version" : "7.10.0",
   "minimum_index_compatibility_version" : "7.0.0"
 },
 "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

configure opensearch

Note: This is a developer setup, not suitable for production, please refer to the opensearch documentation for specific information about securing opensearch

• disable ssl

for testing it is convenient to be able to easily see the communication between samba and opensearch unencrypted, of course ssl can be re-enabled after a working setup has been established.

in /etc/opensearch/opensearch.yml:

-plugins.security.ssl.http.enabled: false
+plugins.security.ssl.http.enabled: true

• allow fscrawler to talk to opensearch

Add following line to /etc/opensearch/opensearch.yml

compatibility.override_main_response_version: true (to allow fscrawler to communicate with opensearch)

Use fscrawler to index files

using latest fscrawler version 2.10 (at time of writing)

• identify (or create) some locations on the filesystem (which are accessible from samba shares) that have content you would like to index
• install fscrawler

unzip fscrawler-distribution-2.10-20240702.144319-374.zip

• create a user to use to communicate with opensearch to populate the index

Here we will use the 'admin' user that comes already setup with opensearch. Note: the opensearch 'admin' user is a super user. You might want to consider creating a specific opensearch user for fscrawler to use that has appropriate roles assigned. e.g. with 'just enough' privileges to access the index(s) you want to create/modify. (see here and associated documentation)

• use fscrawler to create an index

 ./fscrawler-distribution-2.10-SNAPSHOT/bin/fscrawler index_name

you will be prompted (if this is the first time to run the command)

INFO  [f.console] job [index_name] does not exist
INFO  [f.console] Do you want to create it (Y/N)?
answer 'Y'

• edit the config file ~/.fscrawler/index_name/_settings.yaml created in the last step

• configure path fscrawler to index

url: "/path/to/index"

• setup fscrawler to disable ssl when communicating with opensearch

- url: "https://127.0.0.1:9200"
- url: "http://127.0.0.1:9200"

-  ssl_verification: true
+  ssl_verification: false

• don't stop on error (otherwise any problem indexing a specific file will stop the indexing process)

-  continue_on_error: false
+  continue_on_error: true

• setup optional stuff

-  attributes_support: false
-  raw_metadata: false
+  attributes_support: true
+  raw_metadata: true

• run fscrawler again

./fscrawler-distribution-2.10-SNAPSHOT/bin/fscrawler index_name --username admin --loop 1

Configure WSP for samba

use the following global configuration

wsp backend = elasticsearch
elasticsearch:auth=credfile
elasticsearch:auth_file=/etc/samba/usercreds.txt
elasticsearch:wsp_acl_filtering=true

use the following share configuration

wsp = true
elasticsearch:index = index_name
elasticsearch:max results = 200

credfile format

samba_user:opensearch_user%password

'*' can be used in place of a 'samba user' to match all currently unmatched samba users

example

*:admin%1234?Changeme

will map all previously unmapped (in credfile) users to admin note: as the credentials are stored in a local file (which should be root rw only) the opensearch users defined in the credfile should have the most restrictive privileges possible (and no write permissions)

start samba

systemctrl start smb.service

use wspsearch cli or windows client to search for content (e.g. pictures)

wspsearch -U$user%$password //$host/$share --kind picture