LDAP AND SAMLIN HUE
Abraham Elmahrek
WHATIS HUE?
WEB INTERFACE FOR MAKING HADOOP EASIER TO USE Suite of apps for each Hadoop component, like Hive, Pig, Impala, Oozie, Solr, Sqoop2, HBase...
VIEW FROM30K FEET
Hadoop Web Server You and eventhat friend
that uses IE9 ;)
ECOSYSTEM
PIGJO
B BROWSER
JOB DESIG
NER
OOZIE
HIVE IMPA
LA
METASTO
RE BROWSERSEARCH
HBASE BROWSER
SQOOP
ZOOKEEPERUSER ADMIN
DB QUERY
SPARK
HOME ...
GUI DESIG
N
FILE BROWSER
USER
USER WORKFL
OWS
USER
YARN JobTracker Oozie
Pig
HDFS
HiveServer2
Hive Metastore
Cloudera Impala
Solr
HBase
Sqoop2
Zookeeper
LDAP SAML
Hue Plugins
APPS
TARGETOF HUE
GETTING STARTED WITH HADOOP BEING PRODUCTIVE EXPLORING DIFFERENT ANGLES OF THE PLATFORM !
LET ANY USER FOCUS ON BIG DATA PROCESSING
THE CORETEAM PLAYERS
team.gethue.com
ABRAHAM ELMAHREK
ROMAIN RIGAUX
ENRICO BERTI
CHANG BEER
HISTORY
HUE 1
Desktop-like in a browser, did its job but pretty slow, memory leaks and not very IE friendly but definitely advanced for its time (2009-2010).
HISTORY
HUE 2
The first flat structure port, with Twitter Bootstrap all over the place.
HISTORY
HUE 2.5
New apps, improved the UX adding new nice functionalities like autocomplete and drag & drop.
HISTORY
HUE 3 ALPHA
Proposed design, didn’t make it.
HISTORY
HUE 3
Transition to the new UI, major improvements and new apps.
HISTORY
HUE 3.5+
Where we are now, new UI, several new apps, the most user friendly features to date.
LDAP
INTRO
1.Hierarchical entries 2.Entries contain attributes 3.Attributes available are defined
by object classes
TWO KINDS OF PROBLEMS
DIRECT BIND
Authenticate against a directory service using simple direct bind.
SEARCH
Authenticate, import, synchronize, etc. against an LDAP service by searching for a particular entry
EXISTING FEATURES LOGIN
Authenticate against a directory service using simple direct bind or search for the distinguished name to bind with.
USERADMIN
Add new users and groups; Synchronize existing users and groups; Support posix accounts, posix groups, DN import, general LDAP search, etc.
CLI
Command line interface for synchronizing LDAP users and groups.
SUBGROUPS
Import subgroups and members of subgroups when synchronizing a group. Subgroup defined as a subordinate group.
LOWERCASE
Force usernames to lower case.
CONFIGURABLE
User filter, user name attribute, group filter, group name attribute
NEW FEATURESMULTIDOMAIN
Be able to choose which domain to authenticate against.
NESTED GROUPS
Be able to import nested groups and members of nested groups.
EXAMPLE CONFIGURATIONS - BASIC[[ldap]] [[[ldap_servers]]] [[[[mycompany]]]] base_dn="DC=hue-search,DC=ent,DC=cloudera,DC=com" ldap_url=ldap://hue-search.ent.cloudera.com bind_dn="CN=Directory Manager" bind_password=cloudera !
[[[[[users]]]]] user_filter=“objectclass=Person" user_name_attr=uid !
[[[[[groups]]]]] group_filter="objectclass=groupOfNames"
EXAMPLE CONFIGURATIONS - LDAPS[[ldap]] [[[ldap_servers]]] [[[[mycompany]]]] base_dn="DC=hue-search,DC=ent,DC=cloudera,DC=com" ldap_url=ldaps://hue-search.ent.cloudera.com bind_dn="CN=Directory Manager" bind_password=cloudera ldap_cert=/etc/certs/root-ca-cert.pem !
[[[[[users]]]]] user_filter="objectclass=Person" user_name_attr=uid !
[[[[[groups]]]]] group_filter="objectclass=groupOfNames"
EXAMPLE CONFIGURATIONS - NESTED GROUPS[[ldap]] [[[ldap_servers]]] [[[[mycompany]]]] base_dn="DC=hue-search,DC=ent,DC=cloudera,DC=com" ldap_url=ldap://hue-search.ent.cloudera.com bind_dn="CN=Directory Manager" bind_password=cloudera subgroups=nested !
[[[[[users]]]]] user_filter="objectclass=Person" user_name_attr=uid !
[[[[[groups]]]]] group_filter="objectclass=groupOfNames"
EXAMPLE CONFIGURATIONS - DIRECT BIND[[ldap]] [[[ldap_servers]]] [[[[mycompany]]]] base_dn="DC=hue-search,DC=ent,DC=cloudera,DC=com" ldap_url=ldap://hue-search.ent.cloudera.com bind_dn="CN=Directory Manager" bind_password=cloudera search_bind_authentication=false ldap_username_pattern=“uid=<username>,ou=People,dc=hue-search,dc=ent,dc=cloudera,dc=com” ! [[[[[users]]]]] user_filter=“objectclass=Person” user_name_attr=uid ! [[[[[groups]]]]] group_filter=“objectclass=groupOfNames”
EXAMPLE CONFIGURATIONS - ACTIVE DIRECTORY[[ldap]] [[[ldap_servers]]] [[[[mycompany]]]] base_dn="DC=hue-search,DC=ent,DC=cloudera,DC=com" ldap_url=ldap://hue-search.ent.cloudera.com bind_dn="CN=Directory Manager" bind_password=cloudera search_bind_authentication=false nt_domain=cloudera.com ! [[[[[users]]]]] user_filter=“objectclass=Person” user_name_attr=uid ! [[[[[groups]]]]] group_filter=“objectclass=groupOfNames”
EXAMPLE CONFIGURATIONS - ADVANCED[[ldap]] subgroups=nested ignore_username_case=true force_username_lowercase=true ! [[[ldap_servers]]] [[[[mycompany]]]] base_dn="DC=hue-search,DC=ent,DC=cloudera,DC=com" ldap_url=ldap://hue-search.ent.cloudera.com bind_dn="CN=Directory Manager" bind_password=cloudera ! [[[[[users]]]]] user_filter="objectclass=Person" user_name_attr=samaccountname ! [[[[[groups]]]]] group_filter="objectclass=groupOfNames"
SAML
INTRO
1.Service provider (SP) 2. Identity provider (IdP) 3.Signed/encrypted requests and
responses 4. IdP Identity source can be LDAP 5.Secure SSO as defined by the
OASIS group standards
THE CHALLENGESLIBRARIES
Python libraries have bad licenses, are poorly written, and rely on system libraries not found in primary repositories.COMPLEX CONFIGURATION
Service Provider and Identity Provider definition is obscure. Protocol is configurable. Every IdP is slightly different.TESTABILITY
Opensource IdPs are incomplete. We use Shibboleth.
THE BREAK DOWNHACKING
https://github.com/abec/djangosaml2 https://github.com/abec/pysaml2
PACKAGING/CONFIGURATION
Do not package SAML libraries. Instead, require users to install manually. Configure via Hue.
TESTABILITY
We need help!
[libsaml] xmlsec_binary=/opt/local/bin/xmlsec1 entity_id="http://192.168.92.1:8080/saml2/metadata/" metadata_file=/Users/abe/Desktop/idp-metadata.xml key_file=/Users/abe/Desktop/idp.key cert_file=/Users/abe/Desktop/idp.crt
DEPENDENCIESXMLSEC1
Requires xmlsec1 (a nonstandard system package)
DJANGOSAML2
Django application for pysaml2
PYSAML2
Python binding with two implementations: 0.4.x line and 1.x line. 1.x line has had major updates and there is a 2.x line now.
diff --git a/src/saml2/client_base.py b/src/saml2/client_base.pyindex f1aadf3..9206a95 100644--- a/src/saml2/client_base.py+++ b/src/saml2/client_base.py@@ -124,11 +124,7 @@ class Base(Entity): else: setattr(self, foo, False) - # extra randomness- self.allow_unsolicited = self.config.getattr("allow_unsolicited", "sp")- self.artifact2response = {}- self.logout_requests_signed = False # # Private methods@@ -533,8 +529,8 @@ class Base(Entity): if resp is None: return None elif isinstance(resp, AuthnResponse):- #self.users.add_information_about_person(resp.session_info())- #logger.info("--- ADDED person info ----")+ self.users.add_information_about_person(resp.session_info())+ logger.info("--- ADDED person info ----") pass else: logger.error("Response type not supported: %s" % (
INSTALLATIONyum install xmlsec1!
build/env/bin/pip install -e git+https://github.com/abec/pysaml2@HEAD#egg=pysaml2!
build/env/bin/pip install -e git+https://github.com/abec/djangosaml2@HEAD#egg=djangosaml2
USERNAME SOURCEATTRIBUTES
Fetch username for SAML from attributes returned by the IdP
NAMEID
Use transient or persistent Name ID to be username for SAML
[libsaml] … username_source=nameid …
IT’S COMPLICATED
https://wiki.cloudera.com/display/engineering/Hue+and+SAML
FRESH IDEASREPLACE XMLSEC1
Python libraries have bad licenses, are poorly written, and rely on system libraries not found in primary repositories.REPLACE PYSAML2
Pysaml2 doesn’t use intelligent libraries, uses xmlsec1, code base is messy.
SINGLE LOGOUT
Some IdPs provides single logout. Needs to be tested.
DOCUMENTATION
More documentation around all the various IdPs and how to support them is necessary.
TEST ON SITEMINDER
Many customers seem to be using SiteMinder and every IdP is slightly different.
SYSTEM LEVEL TESTS
More system level testing as customers start to use SAML.
DEMO TIME
LINKS
DEMO
http://demo.gethue.com
@gethue
USER GROUP
hue-user@
WEBSITE
http://gethue.com
LEARN
http://learn.gethue.com