Amazing New Features and Fixes

The 0.5.4 release focused on updating the internal code's design and includes all the fixes and changes from the (unreleased) 0.5.3 code base

Some notable updates include native JSON support and the new fiobj dynamic type system.

These include:

Change/Update: (fiobj) The dynamic type library (a version 0.5.3 feature) was redesigned to make it extendable. This means that code that used type testing using a switch statement needs to be rewritten.

Change: minor changes to the versioning scheme removed some version MACROS... this isn't API related, so I don't consider it a breaking change, but it might effect code that relied too much on internal workings. The only valid version macros are the FACIL_VERSION_* macros, in the facil.h header.

Change: (http) the HTTP/1.x parser was re-written and replaced. It should perform the same, while being easier to maintain. Also, the new parser could potentially be used to author an HTTP client.

Change: (websocket) the Websocket parser was re-written and replaced, decoupling the parser and message wrapper from the IO layer. Performance might slightly improve, but should mostly remain the same. However, the new code is easier to maintain and easier to port to other implementations. Also, the new parser supports a client mode (message masking).

Fix: (websocket) issues with the new websocket parser were fixed. Credit to Tom Lahti (@uidzip) for exposing the issues.

Fix: (websocket) fix #16, where a client's first message could have been lost due to long on_open processing times. This was fixed by fragmenting the upgrade event into two events, adding the facil_attach_locked feature and attaching the new protocol before sending the response. Credit to @madsheep and @nilclass for exposing the issue and tracking it down to the on_open callbacks.

Fix: (sock) sockets created using the TCP/IP sock library now use TCP_NODELAY as the new default. This shouldn't be considered a breaking change as much as it should be considered a fix.

Fix: (http1) HTTP/1.x is now more fragmented, avoiding a read loop to allow for mid-stream / mid-processing protocol upgrades. This also fixes #16 at it's root (besides the improved websocket callback handling).

Fix: (http1) HTTP/1.x now correctly initializes the udata pointer to NULL fore each new request.

Fix: (facil) the facil_run_every function now correctly calls the on_finish callback when a timer initialization fails. This fixes a leak that could have occurred due to inconsistent API expectations. Workarounds written due to this issue should be removed.

Fix: (facil) connection timeout is now correctly ignored for timers.

Fix: (defer) a shutdown issue in defer_perform_in_fork was detected by @cdkrot and his fix was implemented.

Fix: (evio) fixed an issue where the evented IO library failed to reset the state indicator after evio_close was called, causing some functions to believe that events are still processed. Now the evio_isactive will correctly indicate that the evented IO is inactive after evio_close was called.

Fix: (evio) fixes an issue where evio_add_timer would fail with EEXIST instead of reporting success (this might be related to timer consolidation concerns in the Linux kernel).

Fix: (evio) better timer fd creation compatibility with different Linux kernels.

Fix: (documentation) credit to @cdkrot for reporting an outdated demo in the README.

Fix: (linking) added the missing -lm linker flag for gcc/Linux (I was using clang, which automatically links to the math library, so I didn't notice this).

Portability: added extern "C" directive for untested C++ support.

Feature: 🎉 added a CLI helper service, allowing easy parsing and publishing of possible command line arguments.

Feature: 🎉🎉 added a dynamic type library to facil.io's core, making some common web related tasks easier to manage.

Feature: 🎉🎉🎉 added native JSON support. JSON strings can be converted to fiobj_s * objects and fiobj_s * objects can be rendered as JSON! I'm hoping to get it benchmarked publicly.

Tiny patches and big plans

Change: non-breaking changes to the folder structure are also reflected in the updated makefile and .clang_complete.

Fix: (defer) Fixed SIGTERM handling (signal was mistakingly filtered away).

Fix: (http_response) Fixed http_response_sendfile2 where path concatenation occurred without a folder separator (/) and exclusively safe file paths were being ignored (the function assumed an unsafe path to be used, at least in part).

Fix: minor fixes and documentation.

Fix / Feature: (facil) sibling processes will now detect a sibling's death (caused by a crashed process) and shutdown.

Feature: @benjcal suggested the script used to create new applications. The current version is a stand-in draft used for testing.

Feature: Temporary boiler plate code for a simple "hello world" HTTP application using the new application script (see the README). This is a temporary design to allow us to test the script's functionality and decide on the final boiler plate's design.

Taking care of the little things.

Fix: (sock) Fixed an issue where sock_flush would always invoke sock_touch, even if no data was actually sent on the wire.

Fix: (sock) fixed a possible issue with sock_flush_strong which might have caused the facil.io to hang.

Fix: (Websocket) fixed an issue with fragmented pipelined Websocket messages and the security patch that was used to prevent such cases from monopolizing resources.

Feature: (facil) easily force an IO event, even if it did not occur, using facil_force_event.

Pack a punch and rock the world 🎉

Braking changes: (pubsub) The API was changed / updated, making pubsub_engine_s objects easier to author and allowing allocations to be avoided by utilizing two void * udata fields... Since this is a breaking change, and following semantic versioning, the minor version is updated. I do wish I could have delayed the version bump, as the roadmap ahead is long, but it is what it is.

Braking changes: (facil) Since the API is already changing a bit, I thought I'd clean it up a bit and have all the on_X flow events (on_close, on_fail, on_start...) share the same function signature where possible.

Changes: (facil) Minor changes to the fio_cluster_* API now use signed message types. All negative msg_type values are reserved for internal use.

Fix: plugging memory leaks while testing the system under very high stress.

Fix: (pubsub, fio_dict) Fixed glob pattern matching... I hope. It seems to work fine, but I'm not sure it the algorithm matches the Redis implementation which is the de-facto standard for channel pattern matching.

Security: (http) the HTTP parser now breaks pipelined HTTP requests into fragmented events, preventing an attacker from monopolizing requests through endless pipelining of requests that have a long processing time.

Fix: (http) http_listen will now always copy the string for the public_folder, allowing dynamic strings to be safely used.

Fix: (http) default error files weren't located due to missing / in name. Fixed by adjusting the requirement for the / to be more adaptive.

Fix: (http) dates were off by 1 day. Now fixed.

Fix: (http1) a minor issue in the on_data callback could have caused the parser to crash in rare cases of fragmented pipelined requests on slower connections. This is now fixed.

Fix?: (http) When decoding the path or a request, the + sign is now left unencoded (correct behavior), trusting in better clients in this great jungle.

Fix: (facil) facil_defer would leak memory if a connection was disconnected while a task was scheduled.

Fix: (facil) facil_connect now correctly calls the on_fail callback even on immediate failures (i.e. when the function call was missing a target address and port).

Fix: (facil) facil_connect can now be called before other socket events (protected form library initialization conflicts).

Fix: (facil) facil_listen will now always copy the string for the port, allowing dynamic strings to be safely used when FACIL_PRINT_STATE is set.

Fix: (facil) facil_last_tick would crash if called before the library was initialized during socket operations (facil_listen, facil_attach, etc')... now facil_last_tick falls back to time() if nothing happened yet.

Fix: (facil) .on_idle now correctly checks for non networks events as well before the callback is called.

Fix: (defer) A large enough (or fast enough) thread pool in a forked process would complete the existing tasks before the active flag was set, causing the facil.io reactor to be stranded in an unscheduled mode, as if expecting to exit. This is now fixed by setting a temporary flag pointer for the forked children, preventing a premature task cleanup.

Changes: Major folder structure updates make development and support for CMake submodules easier. These changes should also make it easier to push PRs for by offering the dev folder for any localized testing prior to submitting the PR.

Feature: (websockets) The websocket pub/sub support is here - supporting protocol tasks as well as direct client publishsing (and autu text/binary detection)! There are limits and high memory costs related to channel names, since pubsub uses a trie for absolute channel matching (i.e. channel name length should be short, definitely less than 1024Bytes).

Feature: (redis) The websocket pub/sub support features a shiny new Redis engine to synchronize pub/sub across machines! ... I tested it as much as I could, but I know my tests are as buggy as my code, so please test before using.

Feature: (facil_listen, http_listen) supports an optional on_finish_rw callback to clean-up the rw_udata object.

Feature: (pubsub) channels now use the available fio_dict_s (trie) data store. The potential price of the larger data-structure is elevated by it's absolute protection against hash collisions. Also, I hope that since channels are more often searched than created, this should improve performance when searching for channels by both pattern and perfect match. I hope this combination of hash tables (for client lookup) and tries (for channel traversal) will provide the best balance between string matching, pattern matching, iterations and subscription management.

Feature: (http) http_listen now supports an on_finish callback.

Feature: (http1) HTTP/1.1 will, in some cases, search for available error files (i.e. "400.html") in the public_folder root, allowing for custom error messages.

Feature: CMake inclusion. Credit to @OwenDelahoy (PR#8).

To use facil.io in a CMake build you may add it as a submodule to the project's repository.

  git submodule add https://github.com/boazsegev/facil.io.git

Then add the following line the project's CMakeLists.txt

  add_subdirectory(facil.io)

Optimize: (fio_hash_table) optimize fio_ht_s bin memory allocations.