Skip to content

Teal Blauwbok: Centipede
Compare
Choose a tag to compare
@funilrys funilrys released this 04 Sep 23:01
· 1246 commits to master since this release
3.3.0
834993f
This commit was signed with the committer’s verified signature.
funilrys Nissar Chababy
GPG key ID: 0D8BFEF5515C00C6
Learn about vigilant mode.

🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟

Hello, World!

I hope that everything is going right for you and your loved one.
This release just provides some improvements and a few new features.

There are a few important changes to notice. The first one is the switch from raw SQL to SQLAlchemy. The reason behind the switch is a change in the development process. I cannot endorse nor want to maintain raw SQL anymore.
To understand the matter, before the switch to SQLAlchemy I had code blocks that handled MySQL connections and statements and another one to handle MariaDB connections and statements. Most of the time it was 1 block for both but in other - not so peaceful - parts there were 2. Meaning that I was handling every use case by myself. With SQLAlchemy, we have 1 line/block for multiple database types. Of course we only support MariaDB and MySQL but it simplifies the integration of other database types in the future. The other reason which convinced me that it's the right way is that it forced me to review and simplify the table structure. Meaning that:

  • We minimize the total size,
  • We optimize the SQL flow,
  • We minimize the number of SQL queries being made.

To simplify the switch and keep it as smooth as possible, a migrator is integrated into PyFunceble. That way, you don't have to take any special action, PyFunceble will handle everything.

The second important change is the sorting algorithms that were improved. Meaning that I simplified them to be efficient but before anything
correct. This change has been made because the sorting of digits was not human friendly. So I took advantage of the issue to completely improve the sorting algorithms.

The third important change is the introduction of new special rules which I let you discover through the documentation.

The fourth and last important change is the new --store-whois argument and logic. Before the introduction of this logic, we were storing the whois WHOIS record into the database. From now, it's only if it is needed by you. The idea behind this is to reduce the size of our database. Please note that the database migrator will follow this argument too.

There is a lot more but that's it for the most important changes!

Let me know what you think or how you use PyFunceble with #PyFunceble or @PyFunceble on Twitter 😄!

I wish you all the best.
Stay safe and healthy!

🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟🎉 🌟

What was fixed ?

  • Typos.
  • Badges URL.
  • An issue under macOS while using the --multiprocess argument.

What was removed ?

  • The db_types directory as it is now obsolete.

What was improved ?

  • Some sections of the documentation.
  • The standard sorting of file content.
  • The hierarchical sorting of the file content.
  • The cleanup logic behind the --clean and --clean-all arguments.
  • The way we process the mining of information.
  • The way we open and close a database (MariaDB/MySQL) connection.

What is new ?

  • The usage of SQLAlchemy instead of raw SQL.
    • With an automatic migration routine if the old layout is detected.
  • The *.wix.com special rule.
  • The *.hpg.com.br special rule.
  • The *.000webhostapp.com special rule.
  • The *.angelfire.com special rule.
  • Introduction of the --store-whois argument and logic.
  • A file position tracker which improve the continuation from an interrupted test.
  • The mining capabilities are now available through the --domain and --url arguments.
  • The multiprocessing mode is automatically deactivated if less than 2 processes are allocated/allowed.
  • The --all argument prints everything.
  • We now retry 3 times with a sleep of 3 seconds between each while downloading vital files. If the URL is still not reachable after 3 retries, the system will stop.
  • CI tests with Python 3.6.11, 3.7.8, and 3.8.5.
  • The PYFUNCEBLE_DEBUG and PYFUNCEBLE_DEBUG_ONSCREEN environment variables.

What is coming ?

  • Fix of reported issues.
  • Improvement of the documentation (help welcome!).
  • Introduction of possible and interesting features.

Contributors

Thanks to those awesome people who contributed to this release!

Assets 5
Loading