Merge pull request #14 from blackears/blender_4.2.1LTS

Blender 4.2.1 lts

.gitignore

@@ -11,3 +11,5 @@
 /examples/headless_example/*.blend1
 /examples/headless_example/headless_result.blend
 /compile_phoneme_db/cache
+/source/__pycache__
+/source/wheels

README.md

@@ -1,6 +1,6 @@
 # Parrot Automatic Lipsync for Blender
 
-This is an addon for Blender that lets you automatically generate lip-synced data from audio tracks.  It can be used for both 2D and 3D animations and supports many common languages.  It uses OpenAI's Whisper library to turn spoken audio tracks into words, and then Gruut to turn words into phonemes to create the key frames for your mouth positions.
+Parrot Lipsync is an extension for Blender that lets you automatically generate lip-synced data from audio tracks.  It can be used for both 2D and 3D animations and supports many common languages.  It uses the `allosaurus` library to turn spoken audio tracks into phonemes which are then used to key frame your mouth positions.
 
 [![Video - overview](https://img.youtube.com/vi/VCXyHdBmBwQ/0.jpg)](https://www.youtube.com/watch?v=VCXyHdBmBwQ)
 
@@ -10,12 +10,12 @@ This is an addon for Blender that lets you automatically generate lip-synced dat
 
 ## Installation
 
-Parrot Lipsync relies on you having some other applications already installed on your computer.
+Parrot Lipsync relies on you having `ffmpeg` already installed on your computer.
 
 
 ### Install Ffmpeg
 
-Ffmpeg is a popular library that lets you read and write many popular media files.  It needs to be accessible from the command line.
+[Ffmpeg](https://ffmpeg.org/) is a popular library that lets you read and write many audio/video files.  It needs to be accessible from the command line.
 
 Open a command prompt and type the following to see if it is already installed:
 
@@ -43,41 +43,10 @@ scoop install ffmpeg
 ```
 
 
-### Installing the Parrot Lipsync addon
+### Installing the Parrot Lipsync extension
 
-Now that ffmpeg is installed, you can install Parrot by opening the `Edit > Preferences > Addons` window and clicking the `Install` button.  Browse to the Parrot Lipsync zip file you downloaded and select it.  Finally, make sure the check mark next to `View 3D: Parrot Lipsync` is checked so that the addon is activated.
+Now that ffmpeg is installed, you can install Parrot by opening the `Edit > Preferences > Addons` tab, clicking the downward pointing arrow button in the top right corner and selecting `Install from Disk...`.  Browse to the Parrot Lipsync zip file you downloaded and select it.  Finally, make sure the check mark next to `View 3D: Parrot Lipsync` is checked so that the extension is activated.
 
-### Install Whisper and Gruut
-
-Parrot needs some support libraries to function.  The easiest way to install them is to open Parrot's configuration page in the `Addon` window to reveal the buttons on the bottom.  Press the `Install whisper_timestamped` and `Install gruut` buttons to ensure these necessary libraries are added to your Blender's python packages.  Some languages will require an additional package - for example, if you intend to use the Russian language, make sure to click the `Install gruut-lang-ru` button as well.
-
-![Addon install panel](doc/addon_install_panel.jpg)
-
-
-### Alternate installation procedure for Whisper and Gruut
-
-It is also possible to install Whsiper and Gruut on the command line as well if you don't want to use the installation panel.
-
-Open a terminal window and go into your Blender installation's python directory (on Windows it should be something like `C:\Program Files\Blender Foundation\Blender 4.0\4.0\python\bin`).  Then issue the following commands:
-
-```
-./python.exe -m pip install --upgrade whisper_timestamped
-./python.exe -m pip install --upgrade gruut
-```
-
-For MacOS and Linux, the following should work:
-
-```
-./python -m pip install --upgrade whisper_timestamped
-./python -m pip install --upgrade gruut
-```
-
-Some languages will require additional packages to be installed:
-
-```
-Russian:
-./python.exe -m pip install --upgrade gruut-lang-ru
-```
 
 ## Usage
 
@@ -167,13 +136,29 @@ The `groups` section defines each mouth position that your animation will use.
 The `phonemes` section is where you let Parrot know which phoneme symbols belong to which group.  The `code` field is the International Phonetic Alphabetic code for the syllable and the `group` field indicates which mouth pose group the syllable belongs to.  The `class` and `example` fields are not currently used by Parrot but meant to provide usage hints to the user and might be used in a future version of Parrot.
 
 
-## Acknowledgements
+## Building
+
+A `make.py` file is included in the root directory.  You must have downloaded the necessary wheels and put them in `./source/wheels` first.  (The wheels are not included in this repo because they are several gigabytes in size all together and some are platform specific.)  You must also have Blender 4.2 or higher installed and in your path.  
+
+The Parrot Lipsync extension can be built by being in the root directory and running the command
+
+```python ./make.py -e```
 
-Whisper AI
-https://github.com/openai/whisper
+This will place the built zipfiles in the `./extensions` directory.
+
+### Getting the wheels
+
+A wheel is an archive of a python module that can be imported into a Python program.  Since Blender 4.2, all necessary wheels must be bundled with the extension.  Since Parrot uses artificial intelligence, it requires several big wheels.  If you run the make file with the `-w` flag, it will download the wheels necessary for the platform you are running on.  
+
+```python ./make.py -w```
+
+There is currently no way (that I know of) to download the wheels for a different platform, but if you want the wheels for platforms other than the one you are building on, you can run the script on those other platforms and then copy the downloaded wheels into your `./source/wheels` directory.
+
+
+## Acknowledgements
 
-Gruut
-https://rhasspy.github.io/gruut/index.html
+allosaurus
+https://github.com/xinjli/allosaurus
 
 Blender Base Meshes:
 https://studio.blender.org/training/stylized-character-workflow/base-meshes/

art/audio/test_pocketsphinx.py

@@ -0,0 +1,19 @@
+from pocketsphinx import AudioFile
+
+for phrase in AudioFile("result.wav"): 
+#for phrase in AudioFile("goforward.raw"): 
+    print(phrase)
+
+
+
+# from os import path 
+# from pydub import AudioSegment 
+
+# # assign files 
+# input_file = "amores_17_lawrence_128kb.mp3"
+# output_file = "result.wav"
+
+# # convert mp3 file to wav file 
+# sound = AudioSegment.from_mp3(input_file) 
+# sound.export(output_file, format="wav") 
+

build_extension.txt

@@ -0,0 +1,3 @@
+blender --command extension build --source-dir source --output-filepath ./export/parrot-1.0.6.zip
+
+

make.py

@@ -15,87 +15,120 @@
 # You should have received a copy of the GNU General Public License 
 # along with this program. If not, see <http://www.gnu.org/licenses/>.
 
-import os
-import shutil
-import sys
 import getopt
 import platform
+import subprocess
+import pip
+import sys
+import os
+import shutil
 
-projectName = 'parrotLipsync'
-
-def copytree(src, dst):
-    for item in os.listdir(src):
+#projectName = 'kitfoxQuadRemesher'
+
+platforms = ["macosx_11_0_arm64", "manylinux_2_28_x86_64", "win_amd64", "manylinux_2_17_x86_64"]
+#platforms = ["macosx_11_0_arm64", "manylinux_2_17_x86_64", "win_amd64"]
+
+python_modules = [
+#"pocketsphinx",
+"allosaurus",
+#"whisper_timestamped-1.15.4",
+#"whisper_timestamped", 
+
+# "gruut", 
+# "gruut-lang-ar",
+# "gruut-lang-cs",
+# "gruut-lang-de",
+# "gruut-lang-en",
+# "gruut-lang-es",
+# "gruut-lang-fa",
+# "gruut-lang-fr",
+# "gruut-lang-it",
+# "gruut-lang-lb",
+# "gruut-lang-nl",
+# "gruut-lang-pt",
+# "gruut-lang-ru",
+# "gruut-lang-sv",
+# "gruut-lang-sw",
+]
+
+# wheels_no_deps = [
+    # "allosaurus", "resampy", "panphon", "requests", "tqdm",    
+    # "filelock", "fsspec", "jinja2", "mpmath", "munkres", "networkx", "panphon", 
+# ]
+
+# wheels_bin = [
+    # "scipy", "torch", "editdistance", 
+    # "editdistance", "llvmlite", "MarkupSafe", "numba", 
+# ]
+
+# wheels_bin = [
+    # ["allosaurus", "--no-deps", "macosx_14_0_arm64"],
+    # ["scipy", "--only-binary=:all:", "macosx_14_0_arm64"],
+# ]
+
+#https://files.pythonhosted.org/packages/07/85/3ef1d2f70736bef5650c7fedbf4d4e01efe728f10bad0294a9a8d396ff00/allosaurus-1.0.2-py3-none-any.whl
+
+#pip download scipy --dest ./wheels-extra-mac --only-binary=:all: --platform=macosx_14_0_arm64
+#pip download scipy --dest ./wheels-extra-mac --only-binary=:all: --platform=macosx_14_0_x86_64
+
+#pip download torch --dest ./wheels-extra-mac --only-binary=:all: --platform=macosx_11_0_arm64
+#pip download editdistance --dest ./wheels-extra-mac --only-binary=:all: --platform=macosx_11_0_arm64
+
+def print_wheel_list():
+    for f in os.listdir("source/wheels"):
+        filename = os.fsdecode(f)
+        print("    \"./wheels/" + filename.strip() + "\",")
+
+def build_wheels():
+    if True:
+        #Currently not working
+        #https://projects.blender.org/blender/blender/issues/127632
+        return
 
-        s = os.path.join(src, item)
-        d = os.path.join(dst, item)
-        if os.path.isdir(s):
-            os.mkdir(d)
-            copytree(s, d)
-        else:
-            filename, extn = os.path.splitext(item)
-            print ("file " + filename + " extn  " + extn)
-            if (extn != ".py" and extn != ".png" and extn != ".json"):
-                continue
-
-            shutil.copy(s, d)
-
-def make(copyToBlenderAddons = False, createArchive = False):
+    curPath = os.getcwd()
+    if os.path.exists('source/wheels'):
+        shutil.rmtree('source/wheels')
+    os.mkdir('source/wheels')
 
-    blenderHome = None
-    # platSys = platform.system()
-    # if platSys == 'Windows':
-        # appData = os.getenv('APPDATA')
-        # blenderHome = os.path.join(appData, "Blender Foundation/Blender/2.92")
-
-    # elif platSys == 'Linux':
-        # home = os.getenv('HOME')
-        # blenderHome = os.path.join(home, ".config/blender/2.92/")
-
+    for plat in platforms:
+        for module in python_modules:
+            #pip.main(["download", module, "--dest", "source/wheels", "--only-binary=:all:", "--python-version=3.11", "--platform=" + plat])
 
-    blenderHome = os.getenv('BLENDER_HOME')
+            subprocess.call(["pip", module, "numba", "-w", "source/wheels2"])
+#            subprocess.call(["python", "-m", "pip", "wheel", "--wheel-dir", "source/wheels", "--only-binary=:all:", module])
+
 
-    #Create build directory
+def build_extension():
     curPath = os.getcwd()
-    if os.path.exists('build'):
-        shutil.rmtree('build')
-    os.mkdir('build')
-    os.mkdir('build/' + projectName)
-
-    copytree("source", "build/" + projectName);
-
+    if os.path.exists('extension'):
+        shutil.rmtree('extension')
+    os.mkdir('extension')
 
-    #Build addon zip file
-    if createArchive: 
-        if os.path.exists('deploy'):
-            shutil.rmtree('deploy')
-        os.mkdir('deploy')
-
-        shutil.make_archive("deploy/" + projectName, "zip", "build")
-
+    subprocess.call(["blender", "--command", "extension", "build", "--split-platforms", "--source-dir", "source", "--output-dir", "extension"])
+
+#pip wheel numba -w .
+#pip download pillow --dest ./wheels --only-binary=:all: --python-version=3.11 --platform=macosx_11_0_arm64
+#pip download pillow --dest ./wheels --only-binary=:all: --python-version=3.11 --platform=manylinux_2_28_x86_64
+#pip download pillow --dest ./wheels --only-binary=:all: --python-version=3.11 --platform=win_amd64 --platform=macosx_11_0_arm64
 
-    if copyToBlenderAddons: 
-        if blenderHome == None:
-            print("Error: BLENDER_HOME not set.  Files not copied to <BLENDER_HOME>/script/addons.")
-            return
-
-        addonPath = os.path.join(blenderHome, "scripts/addons")
-        destPath = os.path.join(addonPath, projectName)
+#blender --command extension build --split-platforms
 
-        print("Copying to blender addons: " + addonPath)
-        if os.path.exists(destPath):
-            shutil.rmtree(destPath)
-        copytree("build", addonPath);
+def install_extension():
+    subprocess.call(["blender", "--command", "extension", "install-file", "--enable", "--repo", "user_default",  "extension/parrot_lipsync-1.1.0-windows_x64.zip"])
 
 
 if __name__ == '__main__':
     copyToBlenderAddons = False
     createArchive = False
 
     for arg in sys.argv[1:]:
-        if arg == "-a":
-            createArchive = True
-        if arg == "-b":
-            copyToBlenderAddons = True
+        if arg == "-w":
+            build_wheels()
+        if arg == "-l":
+            print_wheel_list()
+        if arg == "-e":
+            build_extension()
+        if arg == "-i":
+            install_extension()
 
-    make(copyToBlenderAddons, createArchive)
 

source/__init__.py

@@ -20,17 +20,17 @@
 # SOFTWARE.
 
 
-bl_info = {
-    "name": "Parrot Lipsync",
-    "author": "Mark McKay",
-    "version": (1, 0, 5),
-    "blender": (4, 0, 0),
-    "location": "View3D > Export > Mesh Exporter",
-    "description": "Generate lipsync tracks from audio files using Whisper AI.",
-    #"warning": "",
-    "doc_url": "https://github.com/blackears/parrotLipsync",
-    "category": "View 3D"
-}
+# bl_info = {
+    # "name": "Parrot Lipsync",
+    # "author": "Mark McKay",
+    # "version": (1, 0, 5),
+    # "blender": (4, 0, 0),
+    # "location": "View3D > Export > Mesh Exporter",
+    # "description": "Generate lipsync tracks from audio files using Whisper AI.",
+    # #"warning": "",
+    # "doc_url": "https://github.com/blackears/parrotLipsync",
+    # "category": "View 3D"
+# }
 
 import bpy
 from .parrot_lipsync import ParrotPoseProps
@@ -41,19 +41,9 @@
 from .parrot_lipsync import PLUGIN_OT_ParrotRenderLipsyncToAction
 from .parrot_lipsync import PLUGIN_OT_ParrotRenderLipsyncToRigNla
 from .parrot_lipsync import PLUGIN_OT_ParrotReloadPhonemeTable
-from .parrot_lipsync import ParrotAddonPreferences
-from .parrot_lipsync import LibraryInstaller
-from .parrot_lipsync import LibraryUninstaller
-# from .parrot_lipsync import InstallWhisper
-# from .parrot_lipsync import UninstallWhisper
-# from .parrot_lipsync import InstallGruut
-# from .parrot_lipsync import UninstallGruut
-# from .parrot_lipsync import InstallGruutLangAr
-# from .parrot_lipsync import UninstallGruutLangAr
-# from .parrot_lipsync import InstallGruutLangPt
-# from .parrot_lipsync import UninstallGruutLangPt
-# from .parrot_lipsync import InstallGruutLangRu
-# from .parrot_lipsync import UninstallGruutLangRu
+#from .parrot_lipsync import ParrotAddonPreferences
+#from .parrot_lipsync import LibraryInstaller
+#from .parrot_lipsync import LibraryUninstaller
 
 
 
@@ -67,19 +57,9 @@
     PLUGIN_OT_ParrotRenderLipsyncToAction,
     PLUGIN_OT_ParrotRenderLipsyncToRigNla,
     PLUGIN_OT_ParrotReloadPhonemeTable,
-    ParrotAddonPreferences,
-    LibraryInstaller,
-    LibraryUninstaller,
-    # InstallWhisper,
-    # UninstallWhisper,
-    # InstallGruut,
-    # UninstallGruut,
-    # InstallGruutLangAr,
-    # UninstallGruutLangAr,
-    # InstallGruutLangPt,
-    # UninstallGruutLangPt,
-    # InstallGruutLangRu,
-    # UninstallGruutLangRu,
+#    ParrotAddonPreferences,
+#    LibraryInstaller,
+#    LibraryUninstaller,
 ]
 
 _register, _unregister = bpy.utils.register_classes_factory(_classes)