Fetching the paper…

Reading the bibliography…

2024

LOCR: Location-Guided Transformer for Optical Character Recognition

Sun, Yu, Zhou, Dongzhan, Lin, Chen et al.

Original
PDF
Post on X

Understand

Academic documents are packed with texts, equations, tables, and figures, requiring comprehensive understanding for accurate Optical Character Recognition (OCR).

While end-to-end OCR methods offer improved accuracy over layout-based approaches, they often grapple with significant repetition issues, especially with complex layouts in Out-Of-Domain (OOD) documents.To tackle this issue, we propose LOCR, a model that integrates location guiding into the transformer architecture during autoregression.
We train the model on a dataset comprising over 77M text-location pairs from 125K academic document pages, including bounding boxes for words, tables and mathematical symbols.
LOCR adeptly handles various formatting elements and generates content in Markdown language.

LOCR: Location-Guided Transformer for Optical Character Recognition · Around

Built on

The curious case of neural text degeneration
Original
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019 · 1904
Earlier work this paper cites.
Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension
Original
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019 · 1910
Earlier work this paper cites.
Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression
Original
Zhaohui Zheng, Ping Wang, Wei Liu, Jinze Li, Rongguang Ye, and Dongwei Ren. 2019 · 1911
Earlier work this paper cites.
LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Original
Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, and Ming Zhou. 2019 · 1912
Earlier work this paper cites.
Bleu: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002
Earlier work this paper cites.
Best practices for convolutional neural networks applied to visual document analysis
P.Y. Simard, D. Steinkraus, and J.C. Platt. 2003 · 2003
Earlier work this paper cites.
Meteor: An automatic metric for mt evaluation with improved correlation with human judgments
Satanjeev Banerjee and Alon Lavie. 2005 · 2005
Earlier work this paper cites.
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Original
Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. 2020 · 2006
Earlier work this paper cites.
An overview of the tesseract ocr engine
R. Smith. 2007 · 2007
Earlier work this paper cites.
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Original
Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, and Lidong Zhou. 2020 · 2012
Earlier work this paper cites.
An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition
Original
Baoguang Shi, Xiang Bai, and Cong Yao. 2015 · 2015
Earlier work this paper cites.
Full-Page Text Recognition: Learning Where to Start and When to Stop
Original
Bastien Moysset, Christopher Kermorvant, and Christian Wolf. 2017 · 2017
Earlier work this paper cites.
EAST: An Efficient and Accurate Scene Text Detector
Original
Xinyu Zhou, Cong Yao, He Wen, Yuzhi Wang, Shuchang Zhou, Weiran He, and Jiajun Liang. 2017 · 2017
Earlier work this paper cites.

Similar

Centernet: Keypoint triplets for object detection
Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian. 2019 · 2019
Cited alongside, same era.
Rethinking Text Line Recognition Models
Original
Daniel Hernandez Diaz, Siyang Qin, Reeve Ingle, Yasuhisa Fujii, and Alessandro Bissacco. 2021 · 2021
Cited alongside, same era.
TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
Original
Minghao Li, Tengchao Lv, Jingye Chen, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, and Furu Wei. 2021 · 2021
Cited alongside, same era.
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021 · 2021
Cited alongside, same era.
Scene Text Recognition with Permuted Autoregressive Sequence Models
Original

Then

EfficientOCR: An Extensible, Open-Source Package for Efficiently Digitizing World Knowledge
Original
Tom Bryan, Jacob Carlson, Abhishek Arora, and Melissa Dell. 2023 · 2023
Later among the works it cites.
T-rex: Counting by visual prompting
Original
Qing Jiang, Feng Li, Tianhe Ren, Shilong Liu, Zhaoyang Zeng, Kent Yu, and Lei Zhang. 2023 · 2023
Later among the works it cites.
Segment anything
Original
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023 · 2023
Later among the works it cites.
Visual in-context prompting
Original
Feng Li, Qing Jiang, Hao Zhang, Tianhe Ren, Shilong Liu, Xueyan Zou, Huaizhe Xu, Hongyang Li, Chunyuan Li, Jianwei Yang, et al. 2023 · 2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Darwin Bautista and Rowel Atienza. 2022 · 2022

Cited alongside, same era.

LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking

Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, and Furu Wei. 2022 · 2022

Cited alongside, same era.

Ocr-free document understanding transformer

Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2022 · 2022

Cited alongside, same era.

PP-StructureV2: A Stronger Document Analysis System

Chenxia Li, Ruoyu Guo, Jun Zhou, Mengtao An, Yuning Du, Lingfeng Zhu, Yi Liu, Xiaoguang Hu, and Dianhai Yu. 2022 · 2022

Cited alongside, same era.

Unified structure generation for universal information extraction

Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2022 · 2022

Cited alongside, same era.

Tableformer: Robust transformer modeling for table-text encoding

Jingfeng Yang, Aditya Gupta, Shyam Upadhyay, Luheng He, Rahul Goel, and Shachi Paul. 2022 · 2022

Cited alongside, same era.

DocBed: A Multi-Stage OCR Solution for Documents with Complex Layouts

Wenzhen Zhu, Negin Sokhandan, Guang Yang, Sujitha Martin, and Suchitra Sathyanarayana. 2022 · 2022

Cited alongside, same era.

Nougat: Neural optical understanding for academic documents

Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. 2023 · 2023

Cited alongside, same era.

Doctr: Document transformer for structured information extraction in documents

Haofu Liao, Aruni RoyChowdhury, Weijian Li, Ankan Bansal, Yuting Zhang, Zhuowen Tu, Ravi Kumar Satzoda, R Manmatha, and Vijay Mahadevan. 2023 · 2023

Later among the works it cites.

doctr: Document text recognition

mindee. 2023 · 2023

Later among the works it cites.

Marker: Convert pdf to markdown quickly with high accuracy

Vik Paruchuri and Samuel Lampa. 2023 · 2023

Later among the works it cites.

Recent trends in mathematical expressions recognition: An lda-based analysis

Sakshi Sakshi and Vinay Kukreja. 2023 · 2023

Later among the works it cites.

Exploring ocr capabilities of gpt-4v (ision): A quantitative and in-depth evaluation

Yongxin Shi, Dezhi Peng, Wenhui Liao, Zening Lin, Xinhong Chen, Chongyu Liu, Yuyi Zhang, and Lianwen Jin. 2023 · 2023

Later among the works it cites.

Vary: Scaling up the vision vocabulary for large vision-language models

Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, Jinrong Yang, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. 2023 · 2023

Later among the works it cites.

The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)

Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2023 · 2023

Later among the works it cites.

DocXChain: A Powerful Open-Source Toolchain for Document Parsing and Beyond

Cong Yao. 2023 · 2023

Later among the works it cites.