Auto appending users to /etc/passwd and /etc/group and home directory creation on clients

[berniecase]

Have we considered adding the ability for a script to be created to automate the appending of /etc/passwd and /etc/group on the HDP clusters? This would be separate from the isilon create users/dir script, but it would actually be created by those scripts. Furthermore, what about creating home directories for all those users? It might cut down on some of the log spam I see during service startup, like this:

ambari-agent/ambari-agent.log.2:ERROR 2017-09-08 11:24:53,446 script_alert.py:119 - [Alert][hive_server_process] Failed with result CRITICAL: ['Connection failed on host ambari-3.bernielab.com:10000 (Traceback (most recent call last):\n File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py", line 200, in execute\n check_command_timeout=int(check_command_timeout))\n File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/hive_check.py", line 74, in check_thrift_port_sasl\n timeout=check_command_timeout)\n File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__\n self.env.run()\n File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run\n self.run_action(resource, action)\n File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action\n provider_action()\n File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run\n tries=self.resource.tries, try_sleep=self.resource.try_sleep)\n File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner\n result = function(command, **kwargs)\n File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call\n tries=tries, try_sleep=try_sleep)\n File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper\n result = _call(command, **kwargs_copy)\n File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call\n raise ExecutionFailed(err_msg, code, out, err)\nExecutionFailed: Execution of \'! beeline -u \'jdbc:hive2://ambari-3.bernielab.com:10000/;transportMode=binary\' -e \'\' 2>&1| awk \'{print}\'|grep -i -e \'Connection refused\' -e \'Invalid URL\'\' returned 1. su: warning: cannot change directory to /home/ambari-qa: No such file or directory\nError: Could not open client transport with JDBC Uri: jdbc:hive2://ambari-3.bernielab.com:10000/;transportMode=binary: java.net.ConnectException: Connection refused (state=08S01,code=0)\nError: Could not open client transport with JDBC Uri: jdbc:hive2://ambari-3.bernielab.com:10000/;transportMode=binary: java.net.ConnectException: Connection refused (state=08S01,code=0)\n)']

[dmtucker]

Ya, the problem with producing/appending generated passwd/group files is that Ambari will see that a given user exists and not bother checking/creating the home directory. We should be able to avoid this issue by generating a script with shell commands (e.g. useradd) that can be run on the clients instead. This should be acceptable since the supported OSes for both HDP and CDH are all Linux-based.

I think it's unlikely we would want to run that on the clients automatically, though, because the need for that may vary based on the environment (e.g. if LDAP/AD is involved).

We will address this in an upcoming overhaul/release we have planned 👍