|
17 | 17 | [Docker Hub](https://hub.docker.com/repository/docker/tomasgajarsky/facetorch) [(GPU)](https://hub.docker.com/repository/docker/tomasgajarsky/facetorch-gpu) |
18 | 18 |
|
19 | 19 |
|
20 | | -**Facetorch** is a Python library designed for facial detection and analysis, leveraging the power of deep neural networks. Its primary aim is to curate open-source face analysis models from the community, optimize them for high performance using TorchScript, and integrate them into a versatile face analysis toolkit. The library offers the following key features: |
| 20 | +**Facetorch** is a Python library designed for facial detection and analysis, leveraging the power of deep neural networks. Its primary aim is to curate open-source face analysis models from the community, package them as portable [torch.export](https://pytorch.org/docs/stable/export.html) models, and integrate them into a versatile face analysis toolkit. The library offers the following key features: |
21 | 21 |
|
22 | 22 | 1. **Customizable Configuration:** Easily configure your setup using [Hydra](https://hydra.cc/docs/intro/) and its powerful [OmegaConf](https://omegaconf.readthedocs.io/) capabilities. |
23 | 23 |
|
24 | | -2. **Reproducible Environments:** Ensure reproducibility with tools like [conda-lock](https://github.com/conda-incubator/conda-lock) for dependency management and [Docker](https://docs.docker.com/get-docker/) for containerization. |
| 24 | +2. **Reproducible Environments:** Ensure reproducibility with [uv](https://github.com/astral-sh/uv) for fast Python package management, [conda-lock](https://github.com/conda-incubator/conda-lock) for conda-forge dependency management, and [Docker](https://docs.docker.com/get-docker/) for containerization. |
25 | 25 |
|
26 | | -3. **Accelerated Performance:** Enjoy enhanced performance on both CPU and GPU with [TorchScript](https://pytorch.org/docs/stable/jit.html) optimization. |
| 26 | +3. **Portable Models:** Models are serialized with `torch.export` (`.pt2` format) — no model source code needed at inference time, with dynamic batch support and `torch.compile` compatibility. |
27 | 27 |
|
28 | | -4. **Simple Extensibility:** Extend the library by uploading your model file to Hugging Face Hub (previously Google Drive) and adding a corresponding configuration YAML file to the repository. |
| 28 | +4. **Simple Extensibility:** Extend the library by uploading your model file to Hugging Face Hub and adding a corresponding configuration YAML file to the repository. |
| 29 | + |
| 30 | +5. **Flexible Input:** Accepts file paths, URLs, tensors, numpy arrays, PIL Images, and bytes. Grayscale and RGBA inputs are automatically converted to RGB. |
29 | 31 |
|
30 | 32 | Facetorch provides an efficient, scalable, and user-friendly solution for facial analysis tasks, catering to developers and researchers looking for flexibility and performance. |
31 | 33 |
|
| 34 | +### Requirements |
| 35 | + |
| 36 | +* Python >= 3.10 |
| 37 | +* PyTorch >= 2.0 |
| 38 | + |
32 | 39 | Please use this library responsibly and with caution. Adhere to the [European Commission's Ethics Guidelines for Trustworthy AI](https://ec.europa.eu/futurium/en/ai-alliance-consultation.1.html) to ensure ethical and fair usage. Keep in mind that the models may have limitations and potential biases, so it is crucial to evaluate their outputs critically and consider their impact. |
33 | 40 |
|
34 | 41 |
|
@@ -58,6 +65,28 @@ Check *data/output* for resulting images with bounding boxes and facial 3D landm |
58 | 65 |
|
59 | 66 | (Apple Mac M1) Use Rosetta 2 emulator in Docker Desktop to run the CPU version. |
60 | 67 |
|
| 68 | +### Python API |
| 69 | + |
| 70 | +```python |
| 71 | +from facetorch import FaceAnalyzer |
| 72 | +from omegaconf import OmegaConf |
| 73 | + |
| 74 | +cfg = OmegaConf.load("conf/config.yaml") |
| 75 | +analyzer = FaceAnalyzer(cfg.analyzer) |
| 76 | + |
| 77 | +# Analyze from file path, URL, tensor, numpy array, PIL Image, or bytes |
| 78 | +response = analyzer.run(image_source="path/to/image.jpg") |
| 79 | + |
| 80 | +# Run only specific predictors |
| 81 | +response = analyzer.run(image_source="image.jpg", include_predictors=["fer", "embed"]) |
| 82 | + |
| 83 | +# Skip detector for pre-cropped face inputs |
| 84 | +response = analyzer.run(image_source=face_tensor, skip_detector=True) |
| 85 | + |
| 86 | +# FaceAnalyzer is also callable |
| 87 | +response = analyzer("image.jpg") |
| 88 | +``` |
| 89 | + |
61 | 90 | ### Configure |
62 | 91 |
|
63 | 92 | The project is configured by files located in *conf* with the main file: *conf/config.yaml*. One can easily add or remove modules from the configuration. |
@@ -157,7 +186,7 @@ analyzer |
157 | 186 | 1. CVI-SZU |
158 | 187 | * code: [ME-GraphAU](https://github.com/CVI-SZU/ME-GraphAU) |
159 | 188 | * paper: [Luo et al. - Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition](https://arxiv.org/abs/2205.01782) |
160 | | - * ! Does not work with CUDA > 12.0 |
| 189 | + * Note: As of v1.0.0, the AU model uses torch.export format and works with all CUDA versions |
161 | 190 |
|
162 | 191 | #### Facial Valence Arousal (va) |
163 | 192 |
|
@@ -232,13 +261,22 @@ Run the Docker container: |
232 | 261 |
|
233 | 262 | ### Add predictor |
234 | 263 | #### Prerequisites |
235 | | -1. File of the TorchScript model |
236 | | -2. Repository on Hugging Face Hub for hosting the model (or legacy ID of the Google Drive model file) |
| 264 | +1. Exported `.pt2` model file (see below) |
| 265 | +2. Repository on Hugging Face Hub for hosting the model |
237 | 266 | 3. facetorch [fork](https://docs.github.com/en/get-started/quickstart/fork-a-repo) |
238 | 267 |
|
239 | | -Facetorch works with models that were exported from PyTorch to TorchScript. You can apply [torch.jit.trace](https://pytorch.org/docs/stable/generated/torch.jit.trace.html) function to compile a PyTorch model as a TorchScript module. Please verify that the output of the traced model equals the output of the original model. |
| 268 | +Facetorch uses models exported with [torch.export](https://pytorch.org/docs/stable/export.html) (`.pt2` format). Export your model with dynamic batch support: |
| 269 | + |
| 270 | +```python |
| 271 | +import torch |
240 | 272 |
|
241 | | -Models are now hosted on [Hugging Face Hub](https://huggingface.co/tomas-gajarsky) which is the default download source. You can host your model on your own Hugging Face account or use the legacy Google Drive hosting option by specifying the appropriate downloader in your configuration. |
| 273 | +model.eval() |
| 274 | +batch = torch.export.Dim("batch", min=1, max=64) |
| 275 | +ep = torch.export.export(model, (dummy_input,), dynamic_shapes={"x": {0: batch}}) |
| 276 | +torch.export.save(ep, "model.pt2") |
| 277 | +``` |
| 278 | + |
| 279 | +Verify that the exported model produces the same outputs as the original. Models are hosted on [Hugging Face Hub](https://huggingface.co/tomas-gajarsky). |
242 | 280 |
|
243 | 281 | #### Configuration |
244 | 282 | ##### Create yaml file |
@@ -272,6 +310,13 @@ the requirements of the new model. |
272 | 310 |
|
273 | 311 |
|
274 | 312 | ### Update environment |
| 313 | + |
| 314 | +#### uv (used by Docker dev/test images) |
| 315 | +* Add packages with corresponding versions to ```pyproject.toml``` dependencies |
| 316 | +* Lock the environment: ```uv lock``` |
| 317 | +* Sync the environment: ```uv sync --extra dev``` |
| 318 | + |
| 319 | +#### conda (for conda-forge users) |
275 | 320 | CPU: |
276 | 321 | * Add packages with corresponding versions to ```environment.yml``` file |
277 | 322 | * Lock the environment: ```conda lock -p linux-64 -f environment.yml --lockfile conda-lock.yml``` |
|
0 commit comments