Skip to content

Commit

Permalink
Get ready for release 1.2.1
Browse files Browse the repository at this point in the history
  • Loading branch information
rocky committed Jun 26, 2021
1 parent 411abf1 commit bbdfad4
Show file tree
Hide file tree
Showing 5 changed files with 44 additions and 14 deletions.
10 changes: 10 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
CHANGES
=======

1.2.1
-----

* Add tables for operator precedence.
* Start to add AMSLaTeX symbols. (A future release will finish this)
* Revise ``README.rst``.
* Some small corrections: ``Implies``
* Make ``ujson`` optional


1.2.0
-----

Expand Down
42 changes: 31 additions & 11 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,34 @@

|Packaging status|

Mathics Scanner
===============
Mathics Character Tables and Scanner
=====================================

This is the tokeniser or scanner portion for the Wolfram Language.
This repository really contains two things:

As such, it also contains a full set of translation between Wolfram Language
named characters, their Unicode/ASCII equivalents and code-points.
* extensive tables describing WL symbols and operators their properties
* a tokenizer or scanner portion for the Wolfram Language.

With respect to the first item, there is a commented YAML that contains a
full set of translation between:

* Wolfram Language named characters,
* their Unicode/ASCII equivalents and Unicode and WL code-points,
* Operator name (if symbol is an operator),
* Operator precedence (if an operator)
* Keyboard escape sequences for the symbol

Uses
----

This is used as the scanner inside `Mathics <https://mathics.org>`_ but it can
also be used for tokenizing and formatting Wolfram Language code. In fact we
intend to write one. This library is also quite usefull if you need to work
with Wolfram Language named character and convert them to various formats.
The scanner and character tables are used inside `Mathics <https://mathics.org>`_. However information can
also be used by other programs for tokenizing and formatting Wolfram Language code.

For example, tables are used in `mathics-pygments <https://pypi.org/project/Mathics-Scanner/>`_, a Pygments-based
lexer and highlighter for Mathematica/Wolfram Language source code.

This library may be useful if you need to work with Wolfram Language
named character and convert them to various formats.

Usage
-----
Expand All @@ -30,17 +43,24 @@ Usage
``\[FormalA]``) and Wolfram's internal representation use the
``m̀athics_scanner.characters.named_characters`` dictionary.

To regenerate scanner tables run:
To regenerate JSON-format tables run:

::

$ mathics-generate-json-table

Without options ``mathics-generate-json-table`` produces the maximum set of correspondences.

In most applications though you may need just a few of these. The
``--field`` option can be used to narrow the list of entries to output in JSON. Run
``mathics-generate-json-table --help`` for a full list of fields.


Implementation
--------------

For notes on the implementation of the packages or details on the conversion
scheme please read ``implementation.rst``.
scheme please read `Implementation <https://mathics-scanner.readthedocs.io/en/latest/implementation.html>`_.

Contributing
------------
Expand Down
2 changes: 1 addition & 1 deletion mathics_scanner/data/characters.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion mathics_scanner/version.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@
# well as importing into Python. That's why there is no
# space around "=" below.
# fmt: off
__version__ = "1.2.1.dev0" # noqa
__version__="1.2.1" # noqa
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ def subdirs(root, file="*.*", depth=10):
zip_safe=False,
# metadata for upload to PyPI
maintainer="Mathics Group",
description="A general-purpose computer algebra system.",
description="Characters Tables and Tokenizer for Mathics and the Wolfram Language.",
license="GPL",
url="https://mathics.org/",
keywords=["Mathematica", "Wolfram", "Interpreter", "Shell", "Math", "CAS"],
Expand Down

0 comments on commit bbdfad4

Please sign in to comment.