Correct way of installing and running, documentation state

[cehberlin]

Hi,
I found your package and its seems very feature rich, however I have some problems testing it and it seems as most of the documentation is outdated.

I first tried the tutorial here: https://github.com/asmodehn/rostful/blob/master/doc/tutorial.rst
it was was also linked in another issue. But it seems like not all dependencies are mentioned. I additionally added with "wstool set " catkin_pip and pyros_utils. At this point it is not clear which branches can/should be combined/selected
After successfully building the workspace I was not able to launch rostful.

I receive

/usr/bin/python: No module named webargs.flaskparser; 'rostful' is a package and cannot be directly executed

Here, I am wondering how the submodules under deps are integrated, I could not find anything about them in the code. How are these dependencies resolved?

However, I also tried to use rosinstall in a fresh workspace. Probably this approach is also easier to reproduce.

I used the following extended rosintall.txt

- git: {local-name: rostful, version: master, uri: 'https://github.com/asmodehn/rostful.git'}
- git: {local-name: rostful-examples, version: master, uri: 'https://github.com/asmodehn/rostful-examples.git'}
- git: {local-name: pyros, version: master, uri: 'https://github.com/asmodehn/pyros.git'}
- git: {local-name: pyros-utils, version: devel, uri: 'https://github.com/asmodehn/pyros-utils.git'}

catkin_make succeeded without any problems, but I still have the same issue as above.

BTW: I am using Ubuntu 14.04 with ROS indigo

Thanks a lot for your efforts, I would be glad to get some clarifications on how to install/use your package.

[asmodehn]

Hello,

First thanks for taking the time to try it out :-)

A bit of context :

I am also using Ubuntu 14.04 with ROS Indigo. And my main usecase is to run rostful and pyros form the same catkin workspace, as you did.
In that usecase, dependencies are resolved with rosdep install --from-path src/ --ignore-src -y.

• rostful-examples is probably outdated as I actually don't have enough time to maintain it. Your best bet is to follow the documentation, but there might still be some issues here and there...
• pyros-utils is released as a package on Indigo, so you probably dont need the sources unless you plan to modify something.

Two more things that my be useful to know :

• I am in the process to release rostful as a package into indigo, but this will likely take a bit of time since I have higher priority tasks.
• It is also possible to run rostful directly from a python virtual env, but this is not very well tested or documented yet. I am just mentioning it so that you avoid this (there might be some confusing doc or comments lying around) as a first step to get familiar with it. After you manage to run rostful from a catkin workspace you could investigate how to run it straight from python if it fits your usecase.

As for your specific problem webargs is a pip package, used by rostful, and I made a thirdparty ros release for it : https://github.com/asmodehn/webargs-rosrelease but the package is not available in indigo yet, since I have problems building it : https://travis-ci.org/asmodehn/webargs-rosrelease/jobs/157542425
It seems it is not listed in the package.xml... maybe a rosdistro key would install it from pip directly ?

So this is still a work in progress... I need to build all python pip dependencies as ros packages, as well as reorganize the different branches, and I didn't have time to clean this up yet.

• You could try to retrieve the webargs pip package on your system, and make sure that your python interpreter can find it.
• You could add the missing rosdep keys in the package.xml to get rosdep to install the webargs pip package, waiting for a ros pacakge...
• Also I am currently using the gopher-devel branch from source for my current robot development, so you might want to try this one out... there is currently less dependencies issues and packaging mess there.

Let me know if you encounter any other issue !

[cehberlin]

Hi,
thank you very much for your detailed feedback and sorry for my long delay, I was quite busy in the last weeks. I just followed your recommendations and was able to get the server running with the following steps:

1. mkdir -p new_workspace/src
2. cd src
3. git clone https://github.com/asmodehn/rostful.git
4. [cd rostful]
5. [git checkout master]
6. [cd ..]
7. cd ..
8. rosdep install --from-path src/ --ignore-src -y .
9. catkin_make
10. source devel/setup.bash
11. roslaunch rostful rostful.launch

I have also tried the more recently updated branches gopher_devel and python_mutate, but both fail after roslaunch rostful rostful.launch with following error.

Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 151, in _run_module_as_main
    mod_name, loader, code, fname = _get_module_details(mod_name)
  File "/usr/lib/python2.7/runpy.py", line 109, in _get_module_details
    return _get_module_details(pkg_main_name)
  File "/usr/lib/python2.7/runpy.py", line 101, in _get_module_details
    loader = get_loader(mod_name)
  File "/usr/lib/python2.7/pkgutil.py", line 464, in get_loader
    return find_loader(fullname)
  File "/usr/lib/python2.7/pkgutil.py", line 474, in find_loader
    for importer in iter_importers(fullname):
  File "/usr/lib/python2.7/pkgutil.py", line 430, in iter_importers
    __import__(pkg)
  File "rostful/__init__.py", line 11, in <module>
    from .flaskapp import create_app, setup_app_routes
  File "rostful/flaskapp.py", line 11, in <module>
    from .context import register_teardown
  File "rostful/context.py", line 2, in <module>
    from flask import current_app, g
  File "/usr/local/lib/python2.7/dist-packages/flask/__init__.py", line 21, in <module>
    from .app import Flask, Request, Response
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 27, in <module>
    from . import json, cli
  File "/usr/local/lib/python2.7/dist-packages/flask/cli.py", line 23, in <module>
    class NoAppException(click.UsageError):
AttributeError: 'module' object has no attribute 'UsageError'

So far I was able to follow your tutorial using the master-branch version with the '/chatter' example. However, I got errors using the following generic configuration with regex

    <arg name="topics" default="['/.*']" />
    <arg name="services" default="['/.*']" />

Trace copied from the browser window:

PyrosServiceTimeout: Pyros Service call timed out.

Traceback (most recent call last)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2000, in __call__
return self.wsgi_app(environ, start_response)
File "/opt/ros/indigo/lib/python2.7/dist-packages/flask_reverse_proxy/__init__.py", line 58, in __call__
return self.app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1991, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/opt/ros/indigo/lib/python2.7/dist-packages/flask_restful/__init__.py", line 270, in error_router
return original_handler(e)
File "/opt/ros/indigo/lib/python2.7/dist-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1567, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/opt/ros/indigo/lib/python2.7/dist-packages/flask_restful/__init__.py", line 270, in error_router
return original_handler(e)
File "/opt/ros/indigo/lib/python2.7/dist-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python2.7/dist-packages/flask/views.py", line 84, in view
return self.dispatch_request(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask/views.py", line 149, in dispatch_request
return meth(*args, **kwargs)
File "/home/toffer/Dev/rostful_test2/src/rostful/rostful/flask_views.py", line 190, in get
topics=self.node_client.topics(),
File "/opt/ros/indigo/lib/python2.7/dist-packages/pyros/pyros_client.py", line 207, in topics
six.reraise(PyrosServiceTimeout("Pyros Service call timed out."), None, sys.exc_info()[2])
File "/opt/ros/indigo/lib/python2.7/dist-packages/pyros/pyros_client.py", line 205, in topics
res = self.topics_svc.call(send_timeout=5000, recv_timeout=10000)  # Need to be generous on timeout in case we are starting up multiprocesses
File "/opt/ros/indigo/lib/python2.7/dist-packages/pyzmp/service.py", line 148, in call
Open an interactive python shell in this frameraise ServiceCallTimeout("Did not receive response through ZMQ socket.")

[cehberlin]

Hi, another update: I was just able to use the regex configuration sucessfully in another workspace.

  <arg name="topics" default="['/.*']" />
  <arg name="services" default="['/.*']" />

[asmodehn]

Hello,

Yep, regex should not be a problem, I use it all the time. Glad you got it to work :-)

regarding :

    from flask import current_app, g
  File "/usr/local/lib/python2.7/dist-packages/flask/__init__.py", line 21, in <module>
    from .app import Flask, Request, Response
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 27, in <module>
    from . import json, cli
  File "/usr/local/lib/python2.7/dist-packages/flask/cli.py", line 23, in <module>
    class NoAppException(click.UsageError):
AttributeError: 'module' object has no attribute 'UsageError'

It looks like your flask package is not correct / not proper version...
So :

• This seems to be a flask installed via pip on the system (in usr/local/lib instead of /usr/lib). This should be avoided if possible (as per python guidelines). Any idea how it got there ?
• I wonder which version of flask do you have ?
  In my case :

alexv@asmodehn:~$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import flask
>>> flask.__file__
'/usr/lib/python2.7/dist-packages/flask/__init__.pyc'
>>> flask.__version__
'0.10.1'

[cehberlin]

Hi,
thanks for your fast reply.

You are right, my version was from PIP, I have probably installed it during my tests with your package. I have just uninstalled it (it was Version 0.11.1) and I am now back on the same version as you have. I also experienced the above error from time to time on the master branch, let's see if this is now solved.

Do you recommend the current master or one of the more up-to-date branches?

[cehberlin]

Hi again,
I don't know what actually has happend (in the past I was able to use your package in indigo and kinetic respectively Ubuntu 14.04 and 16.04), but I am struggling again with installing and resolving dependencies with your package. Thus here is what I did in a fresh new empty ROS workspace

cd src
git clone https://github.com/asmodehn/rostful.git
git checkout master
cd ..
catkin_make

This results in

CMake Error at rostful/CMakeLists.txt:15 (catkin_pip_setup):
  Unknown CMake command "catkin_pip_setup".

If I use the most recent branch with

git checkout gopher-devel

I am able to build with catkin_make, but if I am executing roslaunch with

roslaunch rostful rostful.launch 

I get the following error:

...
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 163, in _run_module_as_main
    mod_name, _Error)
  File "/usr/lib/python2.7/runpy.py", line 111, in _get_module_details
    __import__(mod_name)  # Do not catch exceptions initializing package
  File "rostful/__init__.py", line 11, in <module>
    from .flaskapp import create_app, setup_app_routes
  File "rostful/flaskapp.py", line 25, in <module>
    from .api_0_1 import api_blueprint as api_1_blueprint
  File "rostful/api_0_1/__init__.py", line 26, in <module>
    from . import flask_views
  File "rostful/api_0_1/flask_views.py", line 12, in <module>
    import pyros
ImportError: No module named pyros
[rostful-2] process has died [pid 26602, exit code 1, cmd 
...

Can you please guide towards the correct way of using your helpful package?

---

UPDATE:
I just figured it out myself. I forgot to initialize the submodules, not sure if this is suppose to be executed automaticall git submodul update --init and in the package.xml is one dependency missing. You might want to integrate this information into your README.md

<build_depend>pyros</build_depend>
<run_depend>pyros</run_depend>

[asmodehn]

Glad to see you are getting the hang of it :-)

Rostful is quite a highlevel package. It can be used directly from python(virtualenv) or from ROS (I only test indigo).
But because of these 2 usecases, it is important that you understand how your own system works with packages and dependencies, how ROS deals with dependencies (python and system), and how python deals with dependencies. and it is not trivial.

I plan to solidify one way of doing things from source (the python way) and one way of doing things from binary (the ROS way), but it is still some time into the future, because of all the dependencies (pyros, pip pkgs for web, etc) and how they must be refactored and release for both python and ROS. Hopefully when this is done, using rostful will be easier.

My advice is :

• if you want to USE rostful in a ROS system, just use the ROS package. (the last one should be working AFAIK)
• if you want to MODIFY rostful, you should get use to working the python way (since rostful is a pure python package).

I think I will use these guidelines for future versions, mixing python and ROS usecases seems to confuse most people.