Documentation Status PyPI version SWH License

DeLFT

DeLFT (Deep Learning Framework for Text) is a Keras and TensorFlow framework for text processing, focusing on sequence labeling (e.g. named entity tagging, information extraction) and text classification (e.g. comment classification). This library re-implements standard state-of-the-art Deep Learning architectures relevant to text processing tasks.

DeLFT has three main purposes:

  1. Covering text and rich texts: most of the existing Deep Learning works in NLP only consider simple texts as input. In addition to simple texts, we also target rich text where tokens are associated to layout information (font. style, etc.), positions in structured documents, and possibly other lexical or symbolic contextual information. Text is usually coming from large documents like PDF or HTML, and not just from segments like sentences or paragraphs, and contextual features appear very useful. Rich text is the most common textual content used by humans to communicate and work.

  2. Reproducibility and benchmarking: by implementing several state-of-the-art algorithms for both sequence labeling and text classification tasks, we want to offer the capacity to validate reported results and to benchmark several methods under the same conditions and criteria.

  3. Production level, by offering optimzed performance, robustness and integration possibilities, we aim at supporting better engineering decisions/trade-off and successful production-level applications.

Some contributions include:

  • A variety of modern NLP architectures and tasks to be used following the same API.

  • Reduction of the size of RNN models, in particular by removing word embeddings from them. For instance, the model for the toxic comment classifier went down from a size of 230 MB with embeddings to 1.8 MB. In practice the size of all the models of DeLFT is less than 2 MB, except for Ontonotes 5.0 NER model which is 4.7 MB.

  • Implementation of a generic support of categorical features, available in various architectures.

  • Usage of dynamic data generator so that the training data do not need to stand completely in memory.

  • Efficient loading and management of an unlimited volume of static pre-trained embeddings.

  • A comprehensive evaluation framework with the standard metrics for sequence labeling and classification tasks, including n-fold cross validation.

  • Integration of HuggingFace transformers as Keras layers

A native Java integration of the library has been realized in GROBID via JEP.

The latest DeLFT release 0.3.4 has been tested successfully with python 3.8 and Tensorflow 2.9.3. As always, GPU(s) are required for decent training time. For example, a GeForce GTX 1050 Ti (4GB) is working very well for running RNN models and BERT or RoBERTa base models. Using BERT large model is no problem with a GeForce GTX 1080 Ti (11GB), including training with modest batch size. Using multiple GPUs (training and inference) is supported.