Samba4/Andrew and Jelmers Fantasy Page/2010

From SambaWiki
Revision as of 14:09, 26 September 2009 by JelmerVernooij (talk | contribs)

Plans for fortnight ending 26 September

  • Implement clever nTSecurityDescriptor update (Matthieu)
  • Merge Calin's work into Samba-GTK. (Jelmer)
  • Test and Debianize SWAT. (Jelmer)

Achieved so far

  • Merged outstanding patches. (Jelmer)

Plans for fortnight ending 12 September

  • Demonstrate Samba<->Samba replication over DRS (Andrew, tridge)
  • Finally import LDB index patches
  • More work on the SAMLDB module (Matthias)

Achieved so far

  • Worked with tridge to: (Andrew)
    • Add support for linked attribute replication over DRS
    • Fix LDB to be more robust in handling errors in callback-based modules
    • Fix failures on older python installs for the 'dcerpc' tests
    • Rework LDB and Samba4's modules to correctly handle two-stage commits
  • Investigated LDB index performance and proposed patches to fix it
  • Implement correct behavior with supportedEnc field in GetDomainInfo rpc (Matthieu)
  • Refactor rebuildextendeddn so it can be integrated in main repo (Matthieu)

Plans for fortnight ending 29 August

  • Finish basic functions for update script (ie. allow updating at least the schema and adding simple objects) (Matthieu)
  • Push to the central repo (Matthieu)
  • Return full ctr6 structure in dcesrv_drsuapi_DsGetNCChanges (Anatoliy)
  • Start digging in linked attributes (Anatoliy)
  • Test case for "urgent replication" (Kamen)
  • Test case for DsGetNCChanges() (Kamen)

Achieved so far

  • Explanation of Zahari's ACL problem (Andrew)
  • Add and improve ldb python wrappers to assist test and conversion script development (Andrew)
  • Fix 'show_deleted' module not to linearise the search filter (should improve performance) (Andrew)

Plans for fortnight ending 15 August

  • Really start working on a tool for provision update (mainly due to schema update) (Matthieu)
  • Investigate and fix issues with Windows 2008 and Samba4 (as a Windows 2008 level DC) (Andrew)

Achieved so far

  • Review of Matthias's 'Computer information in AD' patch (Andrew)
    • Matthias was finally able to merge his patch!
  • More questions to Microsoft (AES key use) (Andrew)
  • Create a script ( to (re)build extended, usefull for upgrading a long time running setup (Matthieu)

Plans for fortnight ending 1 August

  • Continue investigation on bug 6273 (unable to access windows 2008 share from XP/Samba4) (Matthieu)
  • Start working on a tool for provision update (mainly due to schema update) (Matthieu)
  • Display specifiers (Andrew, Matthias)
  • Prepare for an alpha with vampire capability (Andrew)
  • Add flag to ldb to force canonical form (Andrew)
  • Investigate file server bugs (Andrew)
  • Investigate domain trusts again (Andrew)

Achieved so far

  • Computer informations in AD (Matthias)
  • Nested groups (Matthias)
  • Forwarded question to Microsoft for their comment in Windows 2008 access issue (Andrew)
  • Review of Matthias's 'Computer information in AD' patch (Andrew)
  • Fixed Zahari's segfault in his python wrapper for libnet_ChangePassword (Andrew)
  • Implemented 'net export keytab' to extract a keytab from a Samba4 DC (Andrew)
  • Fixed a number of trivial failures in Samba4's 'make test' (Andrew)
    • This should make real bugs easier to see
  • Fix provision on FreeBSD (Andrew)
  • Find core problem for bug 6273, proposed a patch (Matthieu)

Plans for fortnight ending 18 July

  • Prepare for an alpha with vampire capability (Andrew)
  • Add flag to ldb to force canonical form (Andrew)
    • This is things such as making large 32 bit integers negative, sids always to binary etc
  • Research possibilities how to use Kerberos from within Python code (Zahari)
  • Catch up with Andrew Tridgell on replication (Anatoliy, Kamen)
  • Communicate with Microsoft to establish the correct nTSecurityDescriptors for the partitions in a clean installation, how is the defaultSecurityDescriptor used, how the default DACL of a security token is created, and the function of the extended rights (Nadya)
  • Finish debugging the descriptor inheritance (Nadya)
  • Define tests for descriptor inheritance to be added to unit tests (Nadya)
  • Improve Netlogon dissector in order to drill down on bugs 6272 and 6273 (Matthieu)
  • Investigate the problems with Windows 2008 as a SMB client for Windows XP bug 6272 (Matthieu)

Achieved so far

  • Found the problem for bug 6272, issued a patch that should be integrated by Heimdal (Matthieu)
  • Netlogon dissector of wireshark is now able to decrypt schannel encrypted dialogs, patch sent to samba-technical for comments (Matthieu)
  • Found and fixed python and ldb/talloc issues shown up by nTsecurityDescriptor test by Zahari (Andrew)
  • Fixed Windows7 Join against Samba4 (Andrew)
    • It was failing for the 'add' case.
  • Finalize schemaUpdateNow patch and test(Anatoliy)
    • It does not break possibleInferiors test and the schema update is ok now
    • We should focus on schema consistency checker at some point
  • Make Samba4 report Windows 2008 functional level by default (Andrew)
  • Update to current Heimdal again (as patches have been accepted) (Andrew)
  • Sort out issues with various tests (schemaUpdateNow etc) and get outstanding patches applied (Andrew)
  • Working with community to finally integrate the MS-SNTP signing of NTP replies (Andrew)
  • Discussions with Microsoft to get 'Display specifiers' released under an acceptable licence (Andrew)
    • This should allow an import into Samba4

Plans for fortnight ending 4 July

  • Sort out nTsecurityDescriptor problems from Zahari (Andrew)
  • Work with summer of code students (Andrew)

Achieved so far

  • Worked with tridge to show DRS replication from windows works again (Andrew)
  • Applied patch queue from Matthias (Andrew)

Plans for fortnight ending 20 June

  • Improve automated setup of OpenLDAP backend (Andrew)
  • Finish subunit separation (Jelmer)
  • Maybe WMI..

Achieved so far

  • Samba4 alpha (Andrew)
  • Heimdal merge (Andrew)
  • Fixing Python rpcecho test and Python ldb test
  • Work with Don Davis on Samba4's Kerberos lib requirements (Andrew)

Plans for fortnight ending 6 June

  • rpcecho.python test (Jelmer)
  • Attempt Heimdal merge (Andrew)
  • More work on Kerberos requirements (Andrew)

Achived so far

  • Documentation of Kerberos requirements (in particular requiremnts that a MIT Kerberos swich would require) (Andrew with Don Davis)
  • Fix SAMR tests (Andrew)
  • Fix build with older libnet on Fedora 10
  • LDB performance issues with many users (Andrew and Tridge)
  • Unique indexes in LDB (Andrew and Tridge)
  • Fixed one-level indexes in LDB (Andrew and Tridge)
  • Worked with Howard Chu to chase down nasty crash bugs in OpenLDAP under Samba4's 'make test'

Plans for fortnight ending 23 May

  • Rework Samba4 DC to support only one realm at a time (Andrew)
    • This is not related to trusted domains, but to how we look at our database
  • Fix krbtgt expiry causing kpasswd account to be disabled (Andrew)

Achieved so far

  • 'make test' failures with OpenLDAP backend (Andrew)
    • Reproduced on current code
    • Fedora 11 VM prepared and supplied to Howard Chu for further investigation
  • str_list code (Andrew)
    • str_list_make_v3 added to Samba3 while I was away
    • Investigate why this 'v3' version is required
    • Add unit tests for all aspects of 'common' str_list behaviour
    • Attempt (but not committed) to re-merge all the str_list code

Plans for fortnight ending 9 May

Achieved so far

  • Documentation build system improvements (Jelmer)
    • Changed the docs build system to use dblatex rather than db2latex
    • Remove cruft from docs

Plans for fortnight ending 25 April

  • SambaXP conference
    • Samba4 status report presentation
    • Samba4 and Microsoft presentation

Achieved so far

  • libcli/auth merge (without ldb and Samba3 server-side components) (Andrew)
  • Fix RPC python tests (Andrew, Jelmer)

Plans for fortnight ending 11 April

Achieved so far

  • Use Full WSPP Microsoft schema in Samba4 (Andrew and Tridge)
    • Required a lot of work to make ldb more efficient with a full set of schema
    • Create and test possibleInferiors attribute for AD schema
    • Integrate work by Sreepathi Pai to convert the WSPP schema into LDIF for the provision
  • Prepare merge of charcnv code
    • Required cutting down patch from all code to just sharing a common API

Plans for fortnight ending 28 March

  • Improve the implementation of netr_DsRGetDCNameEx2 (Andrew)
  • Include full AD schema when permitted by Microsoft to do so (Andrew)
  • libcli/auth merge between Samba3 and Samba4 (Andrew)
  • charcvn merge between Samba3 and Samba4 (Andrew)
  • libregistry merge (Jelmer)
  • Samba3 DCE/RPC async (Jelmer)
  • WMI (Jelmer)
  • Fix kpasswd when the krbtgt account has expired (Andrew)

Achived so far

  • Pair programming of restoring minschema to operation
  • Implementation (with Tridge) of UID handling for recursion to a new event context in the VFS layer (Andrew)

Plans for fortnight ending 14 March

  • Improve the implementation of netr_DsRGetDCNameEx2 (Andrew)
  • Include full AD schema when permitted by Microsoft to do so (Andrew)

Achieved so far

  • Proposal for fixes for the 'wrong UID' problem with recursion to a new event context in the VFS layer
  • Improve performance of Samba will a full schema (Andrew)

Plans for fortnight ending 27 February

Achieved so far

  • Release of alpha7 (Andrew)
  • Work on the trusted domains and IPA proposal (Andrew)
  • Remove dependency of GENSEC on the Samba4 auth subsystem (Andrew)
  • Travel plans for SambaXP (Andrew)
  • Work with Microsoft on importing the full AD schema

Plans for fortnight ending 13 February

  • Prepare alpha7
  • Prepare proposal for linking IPA with AD via Samba4 (Andrew)
  • Windows7 join to Samba4
    • Work to add the AES schannel type
    • Fix Samba4 to accept Windows 7 joins

Achieved so far

Plans for fortnight ending 24 January

  • More work reintegrating WMI (Jelmer)
  • Finish full epmapper implementation (Jelmer)
  • Fix random failures of samba4.ldb.python tests (Jelmer)
  • Use subunit in submissions to the buildfarm (Jelmer)

Achieved so far

  • Alpha 6 ! (Andrew, Jelmer)

Plans for fortnight ending 10 January

Plans for fortnight ending 27 December 2008

  • Trusted domains (Andrew)
    • Reproduce metze's sucess trusting a Win2k3 domain
    • Reproduce metze's issue being trusted by a Samba3 domain
  • Make preperations for a alpha release
    • Fixing build farm failures (Andrew and Jelmer)
    • Testing a 'real' deployment (Andrew)
    • Write release notes (Jelmer)

Achieved so far

  • Proper Extended DN support (Andrew)
    • Pushed into the master branch
  • Shared object files for gen_ndr files between Samba 3 and Samba 4 (Jelmer)
  • rewrote SWIG-based Python modules in manual C (Jelmer)
  • made Samba 4 in merged build use shared libraries when possible (Jelmer)
  • fixed several issues building the standalone libraries (Jelmer)
  • prepared Debian package of tevents and packaged new versions of talloc, tdb and ldb (Jelmer)

Plans for fortnight ending 13 December 2008

Achieved so far

  • Added interactive mode to setup/provision (Jelmer)
  • Proper Extended DN support (Andrew)
    • Published final patch to list for review
  • Use Microsoft's full AD Schema in Samba4 (Andrew)
    • Conversion script taken on by Sreepathi Pai
    • Working with Microsoft to correct errors in the schema

Plans for fortnight ending 29 November 2008

Archived so far

  • Proper Extended DN support (Andrew)
    • continued work on implementation and testing

Plans for fortnight ending 15 November 2008

  • Research to check about transitive trusts between AD and MIT realms (Andrew)
  • Proper Extended DN support (Andrew)
    • Needed for Samba3 domain members in a Samba4 domain.
  • Make a Samba4 release
    • Needed for OpenChange, and to give users a solid alpha to test

Achieved so far

  • Increase to tridge's blood pressure (Andrew)
    • Tridge and I worked to learn python and start an 'upgrade_samba4' script to assist users who have to re-provision but do not wish to loose data.
  • Proper Extended DN support (Andrew)
    • Posted initial implementation to mailing list for comment

Plans for fortnight ending 1 November 2008

  • Finish 'unicode' password issues with integration of new charset (Andrew)
    • The character set conversion needs to change invalid sequences to a known 'bad' value
  • Proper Extended DN support (Andrew)
    • Needed for Samba3 domain members in a Samba4 domain.
  • Unique Index support (Andrew)
    • Needed to ensure we don't have more than one 'Administrator' in a domain (for example)
  • Allow registration in endpoint mapper (Jelmer)
  • ncacn_http (Jelmer)
  • Research to check about transitive trusts between AD and MIT realms (Andrew)

Achieved so far

  • Fix kpasswd server to not 'exit(10)' the whole of Samba (Andrew)
    • Found by Apple at the CIFS plugfest
  • Reconciled more library code between Samba 3 and 4 (Jelmer)
    • lib/util
    • librpc/gen_ndr
    • librpc/ndr
  • Repel pstring to nsswitch/ (Jelmer)
  • Move crypt() replacement to libreplace (Jelmer)
  • Enable merged-build automatically in developer builds (Jelmer)
  • Merged Matthias' registry server improvements (Jelmer)
  • Split up selftest code into a Samba4-specific and a generic part (Jelmer)
  • Fix blackbox tests on IPv6-only hosts (Jelmer)
  • Blog posting about interopability with Microsoft (Andrew)

Plans for fortnight ending 18 October 2008

  • Use separate structure for gensec settings (Jelmer)
  • Share DEBUG() code between Samba 3 and Samba 4 (Jelmer)
    • In preparation of merging my libutil-share branch
  • More work getting WMI back to work (Jelmer)

Achieved so far

  • Implement a 'unicode' password pass-down mechanism in LDB
    • This fixes domain trust problems where member servers select a compleatly random password
    • We still need to fix this for kerberos hash types (awating charset work by tridge)

Plans for fortnight ending 4 October 2008

  • Implement a 'unicode' password pass-down mechanism in LDB, or otherwise avoid UCS2 -> UTF8 -> UCS2 problems
  • Trusted domain support (LSA and KDC portions) (Andrew)

Achieved so far

  • Separate out and add tests for Subunit (Jelmer)
  • Remove global_loadparm use in a couple more places (Jelmer)
  • Restructure some of the installation bits together with Matthias (Jelmer)

Plans for fortnight ending 20 september 2008

  • wmi integration (Jelmer)
  • hdb_samba4 (Jelmer)
  • eliminate last EJS (minschema.js, samba3sam.js (Jelmer))
  • Trusted domain support (LSA and KDC portions) (Andrew)

Achieved so far

  • Committed merged build patch to Samba 3 (Jelmer)
  • Made Samba 3 and Samba 4 use the same copy of tdb, talloc, compression, replace, nss_wrapper, socket_wrapper, popt (Jelmer)
  • Committed WMI support to the repository (doesn't compile completely yet though) (Jelmer)
  • Fixed samba3sam.js and removed remaining JavaScript support. (Jelmer)
  • Implemented WSGI standard ( support in web_server.

Plans for fortnight ending 6 september 2008

  • wmi integration (Jelmer)
  • upload samba-gtk into Debian (Jelmer)
  • hdb_samba4 (Jelmer)
  • send out patch for merged franky build (Jelmer)
  • Use franky build for personal Samba4 development (Andrew)
  • eliminate last EJS (minschema.js, samba3sam.js (Jelmer))
  • Trusted domain support (LSA and KDC portions) (Andrew)

Achieved so far

  • Update NTP patch (Andrew)
  • Respond to comments and suggestions on RPMs for Fedora (Andrew)
  • (partial) Trusted domain support (LSA and KDC portions) (Andrew)
  • PAC Verification support over NETLOGON (Andrew)
  • Sent out franky merged build patch, more prerequisites fixed for Franky (Jelmer)

Plans for fortnight ending 23 august 2008

Achieved so far

  • slacking off (Jelmer)
  • Lots of questions to Microsoft on trusted domains and PAC validation (Andrew)
  • Build indexes and attributes directly from the schema, not a hard-coded list (Andrew)
  • Generate the cn=Aggregate schema in Samba4, rather than in minschema.js
    • This prepares us for adding arbitrary schema into Samba4
  • Integrate patches for multi-master OpenLDAP configuration (Andrew)
    • This allows a Samba4 provision-backend to create a multi-master backend, without hand-manipulation by the admin
  • Start of work on trusted domains
    • In our KDC, start with a special case for handling the trusted domains principals
    • In the drsblobs.idl, parse the trustAuthIncoming and trustAuthOutgoing blobs

Plans for fortnight ending 9 August 2008

  • Fix AES compatability with Windows 2008/Vista. (Andrew)
    • It turns out that Metze was starting to chase the same bug
    • The fix is to implement gss_wrap_ex() - ie AEAD, the signing of headers in DCE/RPC packets.
    • Earlier 'use Heimdal for SPNEGO' work is forming a very useful basis for this work
  • Look at smartcard login again (Andrew)
    • Bugs in Dogtag have been allegedly fixed.
  • Trusted domains (Andrew)
    • Add support for trusted domains in our KDC

Achieved so far

Plans for fortnight ending 26 July 2008

Achieved so far

  • Fix LDAP backend to be secure (not anonymous access) (Andrew)
  • Partially Fix vista join bugs due to AES and GSSAPI CFX (Andrew with Tridge)
    • Session keys for smb signing are original length (ie, 32 in this case)
    • Session keys for SAMR encryption are 16 (ie, truncated)
    • Still need to fix GSSAPI encryption for the AES case (it uses AEAD, as seen in NTLM2)
  • Phone calls with Microsoft (Andrew)
    • I now have a regular phone hookup with Microsoft to go over pending issues in the WSPP process
  • Fix 'file not found' errors from clients (Andrew with Tridge)
    • Due to an uninitialised variable, introduced in some recent SMB2 work
    • shows up on systems with extended attributes (typically those using SeLinux, such as Fedora)
    • Perhaps a good reason to push out a new alpha soon

Plans for fortnight ending 12 July 2008

  • wmi integration (Jelmer)
  • upload openchange and samba-gtk into Debian (Jelmer)
  • hdb_samba4 (Jelmer)
  • eliminate last EJS (minschema.js, samba3sam.js)
  • Improve LDAP backend from a technology preview to a deployable system (Andrew)

Achieved so far

  • Continue packaging of OpenChange and Samba4 for Fedora
  • Start work on smart card login (Andrew)
    • Including setting up DogTag certificate system (Andrew)
    • At least to the stage of the first crashes...
  • Rework schema handling to know about auxillary classes (Andrew)
    • Try to do this in common between ad2OLschema and the kludge_acl and objectclass modules.

Plans for fortnight ending 28 June 2008

  • external Heimdal use (Andrew)


  • Created Samba 4 and OpenChange RPM packages (Andrew)
  • test TEST_LDAP=yes (Andrew)
  • Fixed Franky build for odd make versions (Jelmer)

Plans for fortnight ending 14 June 2008

  • Linked attributes for 'net vampire' (Andrew)
  • AES Key support (check with docs and Win2008 on format) in samdb (Andrew)
  • Work to make ldb merge easier for Simo (andrew)
  • Any work required to merge NTP patch with distribution (Andrew)
  • Work with alpha testers on any issues that come up in production deployments of Samba4 (Andrew)

Achieved so far

  • Samba4 alpha4 release (andrew)
    • without LDB merge, which seems a while off yet
  • Sync test with it's (now obsolete) ldap.js predecessor (andrew)
  • Add python bindings for NetBIOS (Jelmer)
  • Improve portability of Franky build (Jelmer)
  • Asked Microsoft about AES key formats (Andrew)
    • Just getting the data from Win2008 failed due to other reasons
  • Continued the battle with Microsoft over NTP documentation (Andrew)
  • Worked on package of Heimdal for Fedora (Andrew)
    • As a preview to packaging Samba4 for Fedora

Plans for fortnight ending 31 May 2008

  • Linked attributes for 'net vampire' (andrew)
  • Make a Samba 4.0 alpha4 release if the ldb branch gets merged (Jelmer)

Achieved so far

  • Implement NTP signing (andrew)
  • Finish CLDAP and NBT netlogon parsing. (Andrew)
    • Including expected value tests (critical to ensuring we return the *right* answer)
    • This should help things like Group Policy, which rely on this 'DC ping' functionality
  • Merge Simo's ldb branch with current v4-0-test (abartlet)
    • Should make Simo's merge task easier.
  • Removed smbpython and restructured Python modules hierarchy to not clutter Python namespace (Jelmer)
  • Merged improvements made by Wilco and Jelmer to the registry during SambaXP (Jelmer)
  • Added documentation to most Python modules and improved descriptions. (Jelmer)
  • Fixed memory bug in autogenerated DCE/RPC Python bindings (Jelmer)
  • Several test infrastructure improvements. (Jelmer)
    • Print full test path for easy inclusion in knownfail lists
    • Make test case name part of test name to allow a test to have different results against different test cases
    • Set PYTHONPATH during test runs
  • Removed unused old EJS DCE/RPC bindings and testscripts (Jelmer)
  • Make it easier to use various libraries externally without including all of Samba 4's build system (Jelmer)
  • Updated Samba 4, OpenChange and Samba-Gtk Debian packages, now passes lintian. (Jelmer)
  • Added Python bindings for IRPC / Messaging interfaces (Jelmer)
    • Rewrote smbstatus in Python
  • Added mechanism for doing "raw" DCE/RPC requests from Python (Jelmer)
    • Also initial work on a script that should attempt to figure out IDL by probing
  • Exposed more DCE/RPC internals from Python bindings (Jelmer)
  • Initial work on WSGI implementation in web_server/ (Jelmer)
  • Added combined buildsystem for Franky

Plans for fortnight ending 17 May 2008

  • Fix our CLDAP netlogon processing to match description in [MS-ADTS] 7.3.3 (andrew)
    • Use this to fix and test group policy handling on Win2000 and WinXP clients

Achieved so far

  • Partial security=server implementation, awaiting VFS proxy merge for testing (Andrew)
  • Removed a large number of dead build farm hosts in response to automated mails (Andrew)
  • Brought back old (D)COM code and made it compile again (Jelmer)
  • Merged GNU make branch (Jelmer)
    • Now allows using system Python with Samba Python modules
  • Finished Samba 4 Debian package together with Christian (Jelmer)
  • Updated Debian packages for OpenChange and Samba-Gtk (Jelmer)
  • Most of the parsing work towards the CLDAP/NBT netlogon consolidation (Andrew)

Plans for fortnight ending 3 May 2008

  • Build Farm improvements
    • See if we can use SQLite to get a bit more done
    • make build farm summary page use sqlite
    • host list, by last reported time
    • last reported time on host individual page
  • Finish security=server re-implementation in Samba4
  • Finish ncacn_http implementation

Achieved so far

  • Very useful Visit to Sam's home company for 2 days
    • Chat with principals to encourage them
    • Jelmer prepared WAFS branch for merging
      • Looks like further development will be upstream, which is great
    • Jelmer did some initial work on tests for proxy code
    • Andrew Started work on 'security=server' re-implementation for Samba4
      • This will allow WAFS to hijack an unsigned connection as a man in the middle attack.
    • Andrew fixed 'make test' to fail if PIDL tests fail
  • Build Farm
    • make build farm send e-mails to dead hosts (based on SQLite database)

Achievements for fortnight ending ending 19 April 2008


  • Successfully gave 3 presentations
    • Samba4 status report (Both)
    • Samba4 and the LDAP backend / Little barber shop of horrors (Andrew)
    • RPC Scripting using Python (Jelmer)
  • Worked with Sam Liddicott
    • He has implemented the start of a WAFS (latency reducing) proxy for Samba4
    • Organised to visit his companies office
  • Improved code coverage to give better 'headline' figure for presentation (Andrew)
    • Working with Kai's winbind work to run metze's structure based tests
    • Kai worked on blackbox tests
    • Required fixing up parts of winbind (untested code is broken code, Andrew)
  • Fixed bugs in Pidl reported by Volker (Jelmer)
  • Added knownfailure support in test code (Jelmer)
  • Split out policy library into separate git repository (Jelmer)
  • Worked with Wilco on more registry tests (Jelmer)
  • Fixed several Python usability bits (Jelmer)
  • Fixed duplication in blackbox tests (Jelmer)
  • Initial work on ncacn_http support (Jelmer)
  • Discussions with Guenther, Michael about reconciling registry, libsmbdotconf and smbdotconf in Samba 3 and 4 (Jelmer)