Improve error message due to bad eapi authentication from bootstrap

[jerearista]

While testing by manually running bootstrap, user saw initially a tuple index out range error

bash-4.3# ./bootstrap
SyslogManager: adding localhost handler
Retrieving config from server
GET http://ztp.vis.kaust.edu.sa:8080/bootstrap/config
Server response to GET config: contents={u'logging': [{u'level': u'DEBUG', u'destination': u'file:/tmp/ztps-log'}, {u'destination': u'logging.vis.kaust.edu.sa:514'}], u'xmpp': {}}
ERROR: Bootstrap process failed because of unknown exception:
 tuple index out of range
<type 'exceptions.IndexError'>
Traceback (most recent call last):
 File "./bootstrap", line 1364, in <module>
 main()
 File "./bootstrap", line 1322, in main
 node = Node(server)
 File "./bootstrap", line 385, in __init__
 self.api_enable_cmds([])
 File "./bootstrap", line 521, in api_enable_cmds
 if exc.args[0] == 32: # Broken PIPE
IndexError: tuple index out of range

This didn't give us much clue as to what was happening so we modified the bootstrap file to print the exception it was catching

try:
    result = self.client.runCmds(1, ['enable'] + cmds, req_format)
except Exception as exc:
    print exc

After this we saw that the issue was due to an authentication error (401 Unauthorized)

bash-4.3# ./bootstrap 
SyslogManager: adding localhost handler
Retrieving config from server
GET http://ztp.vis.kaust.edu.sa:8080/bootstrap/config
Server response to GET config: contents={u'logging': [], u'xmpp': {}}

<ProtocolError for ztps:ztps-password@localhost/command-api: 401 Unauthorized>

ERROR: Bootstrap process failed because of unknown exception:
 'NoneType' object has no attribute '__getitem__'
<type 'exceptions.TypeError'>
Traceback (most recent call last):
 File "./bootstrap", line 1365, in <module>
 main()
 File "./bootstrap", line 1323, in main
 node = Node(server)
 File "./bootstrap", line 385, in __init__
 self.api_enable_cmds([])
 File "./bootstrap", line 530, in api_enable_cmds
 return result[1:]
TypeError: 'NoneType' object has no attribute '__getitem__'​

Now, we've found that the issue was due to the fact that the switch was configured for tacacs authentication

aaa authentication login default group <TACACS_GROUP_NAME> local

This meant that if ztps user isn't in the tacacs user db, we wouldn't be able to login with the ztps user on he switch, unless the tacacs server went down and we could revert to the local authentication method, so we've reverted the preference to

aaa authentication login default group local <TACACS_GROUP_NAME>

and the script started to work as expected.

Now, we know that we won't ever see this issue on a brand new switch out of the box, since it doesn't have a config, but probably there are other people out there like this client that would like to do some tests before and maybe they are using TACACS too and we were thinking that maybe you could add an exception handling for this type of scenario in future releases, so next time if somebody sees this issue it would be easy to troubleshoot, something like this

try:
    result = self.client.runCmds(1, ['enable'] + cmds, req_format)
except Exception as exc:
    print(exc)
    # EOS-4.14.5+: persistent connection might be have been closed
    # on timeout - should recover on first retry
    if exc.errcode == 401:
       print("\nZTPS USER NOT AUTHORIZED, Check your AAA settings\n")
       raise exc
    elif exc.args[0] == 32: # Broken PIPE
       result = self.client.runCmds(1, ['enable'] + cmds, req_format)
    else:
       raise exc