chef-server 12 on RHEL7: chef-server-ctl reconfigure hangs when configuring embedded rabbitmq

[yinym]

Version

chef-server-core-12.0.1-1.x86_64.rpm
Downloaded from https://web-dl.packagecloud.io/chef/stable/packages/el/6/chef-server-core-12.0.1-1.x86_64.rpm

Environment

RHEL 7.0 x86_64

Summary

When I tried to install and configure a chef-server 12 on RHEL7, I got stuck when executing chef-server-ctl. I know current chef-server 12 does not officially support RHEL7. But since it is opensource, maybe we can find a workaround way here.

Recreate Procedure

# chef-server-ctl reconfigure

Log and symptom

# chef-server-ctl reconfigure
the ffi-yajl and yajl-ruby gems have incompatible C libyajl libs and should not be loaded in the same Ruby VM
falling back to ffi which might work (or might not, no promises)
ffi-yajl/json_gem is deprecated, these monkeypatches will be dropped shortly
the ffi-yajl and yajl-ruby gems have incompatible C libyajl libs and should not be loaded in the same Ruby VM
falling back to ffi which might work (or might not, no promises)
ffi-yajl/json_gem is deprecated, these monkeypatches will be dropped shortly
Starting Chef Client, version 11.12.2
Compiling Cookbooks...
Recipe: private-chef::default
  * directory[/etc/opscode] action create (up to date)
  * directory[/etc/opscode/logrotate.d] action create
    - create new directory /etc/opscode/logrotate.d
    - change mode from '' to '0755'
    - change owner from '' to 'root'
    - change group from '' to 'root'
    - restore selinux security context

[2014-12-04T03:34:39-05:00] WARN: Cloning resource attributes for template[/opt/opscode/embedded/service/chef-server-bootstrap/bootstrapper-config/pivotal.yml] from prior resource (CHEF-3694)
[2014-12-04T03:34:39-05:00] WARN: Previous template[/opt/opscode/embedded/service/chef-server-bootstrap/bootstrapper-config/pivotal.yml]: /opt/opscode/embedded/cookbooks/private-chef/recipes/bootstrap.rb:32:in `from_file'
[2014-12-04T03:34:39-05:00] WARN: Current  template[/opt/opscode/embedded/service/chef-server-bootstrap/bootstrapper-config/pivotal.yml]: /opt/opscode/embedded/cookbooks/private-chef/recipes/bootstrap.rb:62:in `from_file'
[2014-12-04T03:34:39-05:00] WARN: Cloning resource attributes for runit_service[redis_lb] from prior resource (CHEF-3694)
[2014-12-04T03:34:39-05:00] WARN: Previous runit_service[redis_lb]: /opt/opscode/embedded/cookbooks/enterprise/definitions/component_runit_service.rb:36:in `block in from_file'
[2014-12-04T03:34:39-05:00] WARN: Current  runit_service[redis_lb]: /opt/opscode/embedded/cookbooks/private-chef/recipes/redis_lb.rb:83:in `from_file'
[2014-12-04T03:34:39-05:00] WARN: Cloning resource attributes for directory[/var/opt/opscode/nginx/etc/addon.d] from prior resource (CHEF-3694)
[2014-12-04T03:34:39-05:00] WARN: Previous directory[/var/opt/opscode/nginx/etc/addon.d]: /opt/opscode/embedded/cookbooks/private-chef/recipes/oc_id.rb:151:in `from_file'
[2014-12-04T03:34:39-05:00] WARN: Current  directory[/var/opt/opscode/nginx/etc/addon.d]: /opt/opscode/embedded/cookbooks/private-chef/recipes/nginx.rb:33:in `block in from_file'
Generating RSA private key, 2048 bit long modulus
.......................................................+++
............+++
e is 65537 (0x10001)
Converging 279 resources
  * link[/usr/bin/private-chef-ctl] action create (up to date)
  * link[/usr/bin/chef-server-ctl] action create (up to date)
  * directory[/etc/opscode] action nothing (skipped due to action :nothing)
  * directory[/etc/opscode/logrotate.d] action nothing (skipped due to action :nothing)
  * log[opscode_webui deprecation notice] action write (skipped due to only_if)
Recipe: private-chef::users
  * user[opscode] action create
    - create user user[opscode]

  * group[opscode] action create
    - alter group group[opscode]
    - replace group members with new list of members

Recipe: private-chef::default
  * file[/etc/opscode/dark_launch_features.json] action create
    - create new file /etc/opscode/dark_launch_features.json
    - update content in file /etc/opscode/dark_launch_features.json from none to 05b75f
        --- /etc/opscode/dark_launch_features.json      2014-12-04 03:34:39.961877633 -0500
        +++ /tmp/.dark_launch_features.json20141204-10966-noalb2        2014-12-04 03:34:39.961877633 -0500
        @@ -1 +1,17 @@
        +{
        +  "quick_start": false,
        +  "new_theme": true,
        +  "private-chef": true,
        +  "sql_users": true,
        +  "add_type_and_bag_to_items": true,
        +  "reporting": true,
        +  "actions": true,
        +  "503_mode": false,
        +  "couchdb_containers": false,
        +  "couchdb_groups": false,
        +  "couchdb_acls": false,
        +  "couchdb_association_requests": false,
        +  "couchdb_organizations": false,
        +  "couchdb_associations": false
        +}
    - change mode from '' to '0644'
    - change owner from '' to 'opscode'
    - change group from '' to 'root'
    - restore selinux security context

  * file[/etc/opscode/webui_pub.pem] action create
    - create new file /etc/opscode/webui_pub.pem
    - update content in file /etc/opscode/webui_pub.pem from none to 191680
        --- /etc/opscode/webui_pub.pem  2014-12-04 03:34:39.988877631 -0500
        +++ /tmp/.webui_pub.pem20141204-10966-1hrnhd6   2014-12-04 03:34:39.989877631 -0500
        @@ -1 +1,10 @@
        +-----BEGIN PUBLIC KEY-----
        +<key content>
        +-----END PUBLIC KEY-----
    - change mode from '' to '0644'
    - change owner from '' to 'root'
    - change group from '' to 'root'
    - restore selinux security context

  * file[/etc/opscode/webui_priv.pem] action create
    - create new file /etc/opscode/webui_priv.pem
    - update content in file /etc/opscode/webui_priv.pem from none to 5c3b78
        --- /etc/opscode/webui_priv.pem 2014-12-04 03:34:40.015877629 -0500
        +++ /tmp/.webui_priv.pem20141204-10966-vh4br9   2014-12-04 03:34:40.015877629 -0500
        @@ -1 +1,28 @@
        +-----BEGIN RSA PRIVATE KEY-----
        +<key content>
        +-----END RSA PRIVATE KEY-----
    - change mode from '' to '0600'
    - change owner from '' to 'opscode'
    - change group from '' to 'root'
    - restore selinux security context

  * file[/etc/opscode/worker-public.pem] action create
    - create new file /etc/opscode/worker-public.pem
    - update content in file /etc/opscode/worker-public.pem from none to c904c5
        --- /etc/opscode/worker-public.pem      2014-12-04 03:34:40.041877627 -0500
        +++ /tmp/.worker-public.pem20141204-10966-1opa5g8       2014-12-04 03:34:40.042877627 -0500
        @@ -1 +1,10 @@
        +-----BEGIN PUBLIC KEY-----
        +<key content>
        +-----END PUBLIC KEY-----
    - change mode from '' to '0644'
    - change owner from '' to 'root'
    - change group from '' to 'root'
    - restore selinux security context

  * file[/etc/opscode/worker-private.pem] action create
    - create new file /etc/opscode/worker-private.pem
    - update content in file /etc/opscode/worker-private.pem from none to faae02
        --- /etc/opscode/worker-private.pem     2014-12-04 03:34:40.057877626 -0500
        +++ /tmp/.worker-private.pem20141204-10966-5gy5ye       2014-12-04 03:34:40.058877626 -0500
        @@ -1 +1,28 @@
        +-----BEGIN RSA PRIVATE KEY-----
        +<key content>
        +-----END RSA PRIVATE KEY-----
    - change mode from '' to '0600'
    - change owner from '' to 'opscode'
    - change group from '' to 'root'
    - restore selinux security context

  * file[/etc/opscode/pivotal.cert] action create
    - create new file /etc/opscode/pivotal.cert
    - update content in file /etc/opscode/pivotal.cert from none to 8e1b81
        --- /etc/opscode/pivotal.cert   2014-12-04 03:34:40.084877624 -0500
        +++ /tmp/.pivotal.cert20141204-10966-1wol606    2014-12-04 03:34:40.084877624 -0500
        @@ -1 +1,22 @@
        +-----BEGIN CERTIFICATE-----
        +<cert content>
        +-----END CERTIFICATE-----
    - change mode from '' to '0644'
    - change owner from '' to 'root'
    - change group from '' to 'root'
    - restore selinux security context

  * file[/etc/opscode/pivotal.pem] action create
    - create new file /etc/opscode/pivotal.pem
    - update content in file /etc/opscode/pivotal.pem from none to a98e47
        --- /etc/opscode/pivotal.pem    2014-12-04 03:34:40.110877622 -0500
        +++ /tmp/.pivotal.pem20141204-10966-dtx8oj      2014-12-04 03:34:40.111877622 -0500
        @@ -1 +1,28 @@
        +-----BEGIN RSA PRIVATE KEY-----
        +<key-content>
        +-----END RSA PRIVATE KEY-----
    - change mode from '' to '0600'
    - change owner from '' to 'opscode'
    - change group from '' to 'root'
    - restore selinux security context

  * directory[/etc/chef] action create
    - create new directory /etc/chef
    - change mode from '' to '0775'
    - change owner from '' to 'root'
    - change group from '' to 'opscode'
    - restore selinux security context

  * directory[/var/opt/opscode] action create (up to date)
  * directory[/var/log/opscode] action create
    - create new directory /var/log/opscode
    - change mode from '' to '0755'
    - change owner from '' to 'opscode'
    - change group from '' to 'opscode'
    - restore selinux security context

  * private-chef_keepalived_safemode[warmfuzzy] action create (skipped due to only_if)
Recipe: enterprise::runit_sysvinit
  * execute[echo 'SV:123456:respawn:/opt/opscode/embedded/bin/runsvdir-start' >> /etc/inittab] action run
    - execute echo 'SV:123456:respawn:/opt/opscode/embedded/bin/runsvdir-start' >> /etc/inittab

  * execute[init q] action run
    - execute init q

  * execute[init q] action nothing (skipped due to action :nothing)
Recipe: private-chef::sysctl-updates
  * execute[sysctl-reload] action nothing (skipped due to action :nothing)
  * bash[dual ip4/ip6 portbind] action run (skipped due to only_if)
Recipe: private-chef::plugins
  * chef_run[recipe[chef-ha-drbd::disable]] action runCompiling Cookbooks...
Converging 0 resources

    - Running chef-run with run list recipe[chef-ha-drbd::disable]

Recipe: private-chef::rabbitmq
  * directory[/var/opt/opscode/rabbitmq] action create
    - create new directory /var/opt/opscode/rabbitmq
    - change mode from '' to '0750'
    - change owner from '' to 'opscode'
    - change group from '' to 'opscode'
    - restore selinux security context

  * directory[/var/opt/opscode/rabbitmq/etc] action create
    - create new directory /var/opt/opscode/rabbitmq/etc
    - change mode from '' to '0750'
    - change owner from '' to 'opscode'
    - change group from '' to 'opscode'
    - restore selinux security context

  * directory[/var/opt/opscode/rabbitmq/db] action create
    - create new directory /var/opt/opscode/rabbitmq/db
    - change mode from '' to '0750'
    - change owner from '' to 'opscode'
    - change group from '' to 'opscode'
    - restore selinux security context

  * directory[/var/log/opscode/rabbitmq] action create
    - create new directory /var/log/opscode/rabbitmq
    - change mode from '' to '0750'
    - change owner from '' to 'opscode'
    - change group from '' to 'opscode'
    - restore selinux security context

  * link[/var/opt/opscode/rabbitmq/db] action create (skipped due to not_if)
  * link[/opt/opscode/embedded/bin/rabbitmqctl] action create (up to date)
  * link[/opt/opscode/embedded/bin/rabbitmq-defaults] action create
    - create symlink at /opt/opscode/embedded/bin/rabbitmq-defaults to /opt/opscode/embedded/service/rabbitmq/sbin/rabbitmq-defaults

  * link[/opt/opscode/embedded/bin/rabbitmq-env] action create (up to date)
  * link[/opt/opscode/embedded/bin/rabbitmq-plugins] action create
    - create symlink at /opt/opscode/embedded/bin/rabbitmq-plugins to /opt/opscode/embedded/service/rabbitmq/sbin/rabbitmq-plugins

  * link[/opt/opscode/embedded/bin/rabbitmq-server] action create (up to date)
  * template[/opt/opscode/embedded/service/rabbitmq/sbin/rabbitmq-defaults] action create
    - update content in file /opt/opscode/embedded/service/rabbitmq/sbin/rabbitmq-defaults from 4bcf8e to 2463be
        --- /opt/opscode/embedded/service/rabbitmq/sbin/rabbitmq-defaults       2014-12-17 14:40:02.000000000 -0500
        +++ /tmp/chef-rendered-template20141204-10966-14wuk75   2014-12-04 03:34:40.525877593 -0500
        @@ -16,7 +16,7 @@
         ##

         ### next line potentially updated in package install steps
        -SYS_PREFIX=${RABBITMQ_HOME}
        +SYS_PREFIX=/var/opt/opscode/rabbitmq

         ### next line will be updated when generating a standalone release
         ERL_DIR=
        @@ -26,12 +26,12 @@

         ## Set default values

        -CONFIG_FILE=${SYS_PREFIX}/etc/rabbitmq/rabbitmq
        -LOG_BASE=${SYS_PREFIX}/var/log/rabbitmq
        -MNESIA_BASE=${SYS_PREFIX}/var/lib/rabbitmq/mnesia
        -ENABLED_PLUGINS_FILE=${SYS_PREFIX}/etc/rabbitmq/enabled_plugins
        +CONFIG_FILE=${SYS_PREFIX}/etc/rabbitmq
        +LOG_BASE=${SYS_PREFIX}/log
        +MNESIA_BASE=${SYS_PREFIX}/db
        +ENABLED_PLUGINS_FILE=${SYS_PREFIX}/etc/enabled_plugins

         PLUGINS_DIR="${RABBITMQ_HOME}/plugins"

        -CONF_ENV_FILE=${SYS_PREFIX}/etc/rabbitmq/rabbitmq-env.conf
        +CONF_ENV_FILE=/var/opt/opscode/rabbitmq/etc/rabbitmq.conf
    - restore selinux security context

  * template[/var/opt/opscode/rabbitmq/etc/rabbitmq.conf] action create
    - create new file /var/opt/opscode/rabbitmq/etc/rabbitmq.conf
    - update content in file /var/opt/opscode/rabbitmq/etc/rabbitmq.conf from none to 40477a
        --- /var/opt/opscode/rabbitmq/etc/rabbitmq.conf 2014-12-04 03:34:40.552877591 -0500
        +++ /tmp/chef-rendered-template20141204-10966-1b46tr8   2014-12-04 03:34:40.553877591 -0500
        @@ -1 +1,7 @@
        +HOME="/var/opt/opscode/rabbitmq"
        +RABBITMQ_MNESIA_BASE="/var/opt/opscode/rabbitmq/db"
        +RABBITMQ_LOG_BASE="/var/opt/opscode/rabbitmq/log"
        +RABBITMQ_NODE_IP_ADDRESS="127.0.0.1"
        +RABBITMQ_NODE_PORT="5672"
        +NODENAME="rabbit@localhost"
    - change mode from '' to '0644'
    - change owner from '' to 'root'
    - change group from '' to 'root'
    - restore selinux security context

  * execute[restart_rabbitmq_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/rabbitmq/config] action create
    - create new file /var/log/opscode/rabbitmq/config
    - update content in file /var/log/opscode/rabbitmq/config from none to c52569
        --- /var/log/opscode/rabbitmq/config    2014-12-04 03:34:40.604877587 -0500
        +++ /tmp/chef-rendered-template20141204-10966-56xbvm    2014-12-04 03:34:40.607877587 -0500
        @@ -1 +1,4 @@
        +# svlogd configuration
        +S104857600
        +n10
    - change mode from '' to '0644'
    - change owner from '' to 'root'
    - change group from '' to 'root'
    - restore selinux security context

Recipe: <Dynamically Defined Resource>
  * service[rabbitmq] action nothing (skipped due to action :nothing)
Recipe: private-chef::rabbitmq
  * runit_service[rabbitmq] action enableRecipe: <Dynamically Defined Resource>
  * directory[/opt/opscode/sv/rabbitmq] action create
    - create new directory /opt/opscode/sv/rabbitmq
    - change mode from '' to '0755'
    - restore selinux security context

  * template[/opt/opscode/sv/rabbitmq/run] action create
    - create new file /opt/opscode/sv/rabbitmq/run
    - update content in file /opt/opscode/sv/rabbitmq/run from none to c7603d
        --- /opt/opscode/sv/rabbitmq/run        2014-12-04 03:34:40.662877583 -0500
        +++ /tmp/chef-rendered-template20141204-10966-15axhlb   2014-12-04 03:34:40.663877583 -0500
        @@ -1 +1,5 @@
        +#!/bin/sh
        +exec 2>&1
        +exec chpst -P -u opscode -U opscode /usr/bin/env HOME=/var/opt/opscode/rabbitmq /opt/opscode/embedded/bin/rabbitmq-server
        +
    - change mode from '' to '0755'
    - restore selinux security context

  * directory[/opt/opscode/sv/rabbitmq/log] action create
    - create new directory /opt/opscode/sv/rabbitmq/log
    - change mode from '' to '0755'
    - restore selinux security context

  * directory[/opt/opscode/sv/rabbitmq/log/main] action create
    - create new directory /opt/opscode/sv/rabbitmq/log/main
    - change mode from '' to '0755'
    - restore selinux security context

  * template[/opt/opscode/sv/rabbitmq/log/run] action create
    - create new file /opt/opscode/sv/rabbitmq/log/run
    - update content in file /opt/opscode/sv/rabbitmq/log/run from none to 0e09b8
        --- /opt/opscode/sv/rabbitmq/log/run    2014-12-04 03:34:40.735877578 -0500
        +++ /tmp/chef-rendered-template20141204-10966-1ogqpsu   2014-12-04 03:34:40.736877578 -0500
        @@ -1 +1,4 @@
        +#!/bin/sh
        +exec chpst -U opscode -u opscode \
        +  svlogd -tt /var/log/opscode/rabbitmq
    - change mode from '' to '0755'
    - restore selinux security context

  * template[/opt/opscode/sv/rabbitmq/log/config] action create
    - create new file /opt/opscode/sv/rabbitmq/log/config
    - update content in file /opt/opscode/sv/rabbitmq/log/config from none to e3b0c4
        (no diff)
    - change mode from '' to '0644'
    - restore selinux security context

  * link[/opt/opscode/init/rabbitmq] action create
    - create symlink at /opt/opscode/init/rabbitmq to /opt/opscode/embedded/bin/sv


    - configure service runit_service[rabbitmq]  * link[/opt/opscode/service/rabbitmq] action create
    - create symlink at /opt/opscode/service/rabbitmq to /opt/opscode/sv/rabbitmq

The process stuck at above forever. I tried to wait for hours, but not progress. I need to interrupt it by CTRL+C. And it reports as:

^C[2015-01-15T21:19:16-05:00] FATAL: SIGINT received, stopping
  * directory[/opt/opscode/sv/rabbitmq] action create (up to date)
  * template[/opt/opscode/sv/rabbitmq/run] action create (up to date)
  * directory[/opt/opscode/sv/rabbitmq/log] action create (up to date)
  * directory[/opt/opscode/sv/rabbitmq/log/main] action create (up to date)
  * template[/opt/opscode/sv/rabbitmq/log/run] action create (up to date)
  * template[/opt/opscode/sv/rabbitmq/log/config] action create (up to date)
  * link[/opt/opscode/init/rabbitmq] action create (up to date)

    - configure service runit_service[rabbitmq]

Recipe: private-chef::rabbitmq
  * execute[/opt/opscode/bin/private-chef-ctl start rabbitmq] action run
================================================================================
Error executing action `run` on resource 'execute[restart_rabbitmq_log_service]'
================================================================================


Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of /opt/opscode/embedded/bin/sv restart /opt/opscode/sv/rabbitmq/log ----
STDOUT: warning: /opt/opscode/sv/rabbitmq/log: unable to open supervise/ok: file does not exist
STDERR:
---- End output of /opt/opscode/embedded/bin/sv restart /opt/opscode/sv/rabbitmq/log ----
Ran /opt/opscode/embedded/bin/sv restart /opt/opscode/sv/rabbitmq/log returned 1


Resource Declaration:
---------------------
# In /opt/opscode/embedded/cookbooks/enterprise/definitions/component_runit_service.rb

 18:   execute "restart_#{component}_log_service" do
 19:     command "#{node['runit']['sv_bin']} restart #{node['runit']['sv_dir']}/#{component}/log"
 20:     action :nothing
 21:   end
 22:



Compiled Resource:
------------------
# Declared in /opt/opscode/embedded/cookbooks/enterprise/definitions/component_runit_service.rb:18:in `block in from_file'

execute("restart_rabbitmq_log_service") do
  params {:package=>"private_chef", :log_directory=>nil, :svlogd_size=>nil, :svlogd_num=>nil, :ha=>nil, :control=>nil, :action=>:enable, :name=>"rabbitmq"}
  action [:nothing]
  retries 0
  retry_delay 2
  guard_interpreter :default
  command "/opt/opscode/embedded/bin/sv restart /opt/opscode/sv/rabbitmq/log"
  backup 5
  returns 0
  cookbook_name :"private-chef"
  recipe_name "rabbitmq"
end




Running handlers:
[2015-01-15T21:20:23-05:00] ERROR: Running exception handlers
Running handlers complete

[2015-01-15T21:20:23-05:00] ERROR: Exception handlers complete
[2015-01-15T21:20:23-05:00] FATAL: Stacktrace dumped to /opt/opscode/embedded/cookbooks/cache/chef-stacktrace.out
Chef Client failed. 33 resources updated in 3692746.209538932 seconds
[2015-01-15T21:20:23-05:00] FATAL: Chef::Exceptions::MultipleFailures: Multiple failures occurred:
* Mixlib::ShellOut::ShellCommandFailed occurred in chef run: execute[/opt/opscode/bin/private-chef-ctl start rabbitmq] (private-chef::rabbitmq line 80) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/opscode/bin/private-chef-ctl start rabbitmq ----
STDOUT: warning: rabbitmq: unable to open supervise/ok: file does not exist
STDERR: the ffi-yajl and yajl-ruby gems have incompatible C libyajl libs and should not be loaded in the same Ruby VM
falling back to ffi which might work (or might not, no promises)
ffi-yajl/json_gem is deprecated, these monkeypatches will be dropped shortly
---- End output of /opt/opscode/bin/private-chef-ctl start rabbitmq ----
Ran /opt/opscode/bin/private-chef-ctl start rabbitmq returned 1
* Mixlib::ShellOut::ShellCommandFailed occurred in delayed notification: execute[restart_rabbitmq_log_service] (private-chef::rabbitmq line 18) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/opscode/embedded/bin/sv restart /opt/opscode/sv/rabbitmq/log ----
STDOUT: warning: /opt/opscode/sv/rabbitmq/log: unable to open supervise/ok: file does not exist
STDERR:
---- End output of /opt/opscode/embedded/bin/sv restart /opt/opscode/sv/rabbitmq/log ----
Ran /opt/opscode/embedded/bin/sv restart /opt/opscode/sv/rabbitmq/log returned 1

It will wait for a while on execute[/opt/opscode/bin/private-chef-ctl start rabbitmq] action run, and then failed.

Similar Issue

1. There is a similar issue reported:
   Failures on RHEL 7 chef-cookbooks/runit#74 (comment)
   However, it is using 11.x chef server
2. Another issue on CentOS 7:
   Chef Server runit install fails on CentOS 7 chef#1955
   It was also using chef server 11.x. And from the output log, it fails to locate /etc/init folder, which does not fit the symptom here.

[yinym]

This issue is caused by enterprise cookbook included in chef-server 12.0.1 (/opt/opscode/embedded/cookbooks/enterprise).

From the log:

Recipe: enterprise::runit_sysvinit
  * execute[echo 'SV:123456:respawn:/opt/opscode/embedded/bin/runsvdir-start' >> /etc/inittab] action run
    - execute echo 'SV:123456:respawn:/opt/opscode/embedded/bin/runsvdir-start' >> /etc/inittab

  * execute[init q] action run
    - execute init q

  * execute[init q] action nothing (skipped due to action :nothing)

Within chef-server-ctl reconfigure came to enterprise cookbook (https://github.com/opscode-cookbooks/enterprise-chef-common/blob/master/recipes/runit.rb#L27), it used runit_sysvinit for RHEL7. However, above command in log did not make the /opt/opscode/embedded/bin/runsvdir-start to work. Therefore, it prevented following embedded services to register to runsv and start. The first one is "rabbitmq". That's why we saw the progress hung on rabbitmq, but that is not the rabbitmq recipe issue.

[yinym]

Opened an issue in enterprise cookbook: chef-cookbooks/enterprise-chef-common#26
Fix to enterprise cookbook issue will be tracked in that issue.

[stevendanna]

This should be fixed in 12.0.3.

[jpoley]

its not fixed in chef-server-core-12.0.5-1.el6.x86_64.rpm

[irab]

Just got this same error trying to install chef-server-core-12.1.2-1.el7.x86_64.rpm inside a centos 7 docker container. For repeatability, Dockerfile is just:

FROM centos
RUN yum -y install wget
RUN wget https://web-dl.packagecloud.io/chef/stable/packages/el/7/chef-server-core-12.1.2-1.el7.x86_64.rpm
RUN rpm -Uvh chef-server-core-12.1.2-1.el7.x86_64.rpm
RUN chef-server-ctl reconfigure

[2015-08-12T05:54:59+00:00] INFO: Processing execute[init q] action run (enterprise::runit_sysvinit line 18)

================================================================================
Error executing action `run` on resource 'execute[init q]'
================================================================================

Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of init q ----
STDOUT:
STDERR: Couldn't find an alternative telinit implementation to spawn.
---- End output of init q ----
Ran init q returned 1

Resource Declaration:
---------------------
# In /opt/opscode/embedded/cookbooks/cache/cookbooks/enterprise/recipes/runit_sysvinit.rb

 18: execute "init q" do
 19:   action :nothing
 20: end

Compiled Resource:
------------------
# Declared in /opt/opscode/embedded/cookbooks/cache/cookbooks/enterprise/recipes/runit_sysvinit.rb:18:in `from_file'

execute("init q") do
  action [:nothing]
  retries 0
  retry_delay 2
  default_guard_interpreter :execute
  command "init q"
  backup 5
  returns 0
  declared_type :execute
  cookbook_name "enterprise"
  recipe_name "runit_sysvinit"
end

[2015-08-12T05:54:59+00:00] INFO: Running queued delayed notifications before re-raising exception
[2015-08-12T05:54:59+00:00] ERROR: Running exception handlers
[2015-08-12T05:54:59+00:00] ERROR: Exception handlers complete
[2015-08-12T05:54:59+00:00] FATAL: Stacktrace dumped to /opt/opscode/embedded/cookbooks/cache/chef-stacktrace.out
[2015-08-12T05:54:59+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: execute[init q] (enterprise::runit_sysvinit line 18) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of init q ----
STDOUT:
STDERR: Couldn't find an alternative telinit implementation to spawn.
---- End output of init q ----
Ran init q returned 1

[irab]

Sorry issue with Centos7, ignore the above comment: moby/moby#13297

[salimkapadia]

Just got the same error on chef-server-core-12.3.0-1.el6.x86_64.rpm

[ephracis]

Got this same error with both 12.4.1 and 12.4.0 on Ubuntu 14.04.

[Ufflo]

Same error for 12.6.0-1 on Ubuntu 16.04

[z-matth]

Same error Chef Server 12.5.0-1 on Ubuntu 14.04.

[stevendanna]

Reopened this as a number of people are seeing the same symptoms still.

[marcparadise]

If these are occurring in Docker images, this is a known limitation - let's open a new issue for that so that we can track if this issue is occurring on supported configurations, or only under Docker.

That being said, if you are seeing this on Docker, there is a workaround that you can try.

Chef Server users runit - and if we tell it to run as a full init system, it takes care of this issue.

To do this, manually fork the runsv-start that's embedded with chef-server. This will permit the runit-enabled services to start successfuly. If you're looking to reconfigure Chef Server after the container is up:

Dockerfile:

ADD /path/to/chef-server-startup.sh /chef-server-startup.sh 
CMD ["bash", "/chef-server-startup.sh"] 

chef-server-startup.sh:

#!/bin/bash
export PATH=/opt/opscode/bin:/opt/opscode/bin/embedded:$PATH

# Start this so that chef-server-ctl sv-related commands can
# interact with its services via runsv
/opt/opscode/embedded/bin/runsvdir-start &
chef-server-ctl reconfigure

# Something useful that also keeps the container running...
tail -f /var/log/opscode/nginx/access.log

It's similar if you want to reconfigure in the Docker image build and start Chef Server in the running container. In this case, your container build will need to do the same thing:

Dockerfile:

ADD chef-server-setup.sh /chef-server-setup.sh
RUN /bin/bash /chef-server-setup.sh
ADD /path/to/chef-server-startup.sh /chef-server-startup.sh 
CMD ["bash", "/chef-server-startup.sh"] 

chef-server-setup.sh:

/opt/opscode/embedded/bin/runsvdir-start &
chef-server-ctl reconfigure
chef-server-startup.sh 
```bash
#!/bin/bash
export PATH=/opt/opscode/bin:/opt/opscode/bin/embedded:$PATH
/opt/opscode/embedded/bin/runsvdir-start &
chef-server-ctl start
# Output current state of things for sanity's sake
chef-server-ctl status
# Something useful that also keeps the container running...
tail -f /var/log/opscode/nginx/access.log

[marcparadise]

In any case - can we get clarification from those currently seeing this @ephracis @z-matth @Ufflo - are you encountering this on a docker or LXC container, or is it on VM/bare metal?

[z-matth]

I'm not on Docker. Using Ubuntu14.04 on regular AWS EC2 VM.

[ephracis]

I am running it on a VM using Vagrant/VirtualBox.

I believe my case was caused by the VM running low on memory. I bumped it from the default 2 Gb to 4 Gb and it I haven't had any issues since.

[stevendanna]

Hi Everyone,

Sorry you've had trouble getting chef-server installed. The originally reported issue is likely fixed (we build and test full installs of chef-server on RHEL 7 with every commit). I don't doubt, however, that many of you have seen similar looking error messages. I'm going to close this out, but if you are still seeing something similar, please open a new bug report with:

• The operating system you are running on
• A bit of information about the environment (docker/no-docker, system memory, cpu, etc), and
• The full error output that you are seeing

[manrajesh]

FYI, Got the same ERROR when trying to install chef-server-core-12.13 on EC2 micro instance with RHEL7.3, 1 CPU and 1 GB RAM and later Re-installed on EC2 Medium sized with 2 CPU and 4GB RAM which worked perfectly without any trouble

[Johnjosh11]

I have tried this on docker, but still its not working. can some one please help me on this.

export PATH=/opt/opscode/bin:/opt/opscode/bin/embedded:$PATH

Start this so that chef-server-ctl sv-related commands can

interact with its services via runsv

/opt/opscode/embedded/bin/runsvdir-start &
chef-server-ctl reconfigur