2019

Image-based table recognition: data, model, and evaluation

Zhong, Xu, ShafieiBavani, Elaheh, Yepes, Antonio Jimeno

Understand

Important information that relates to a specific topic in a document is often organized in tabular format to assist readers with information retrieval and comparison, which may be difficult to provide in natural language.

However, tabular data in unstructured digital documents, e.g., Portable Document Format (PDF) and images, are difficult to parse into structured machine-readable format, due to complexity and diversity in their structure and style.
To facilitate image-based table recognition with deep learning, we develop the largest publicly available table recognition dataset PubTabNet (https://github.com/ibm-aur-nlp/PubTabNet), containing 568k table images with corresponding structured HTML representation.
PubTabNet is automatically generated by matching the XML and PDF representations of the scientific articles in PubMed Central Open Access Subset (PMCOA).

Built on

V. I. Levenshtein, “Binary codes capable of correcting deletions, insertions, and reversals,” in Soviet physics doklady , vol. 10, no. 8, 1966, pp. 707–710
1966
Earlier work this paper cites.
Y. Hirayama, “A method for table structure analysis using dp matching,” in Proceedings of 3rd International Conference on Document Analysis and Recognition , vol. 2. IEEE, 1995, pp. 583–586
1995
Earlier work this paper cites.
S. Tupaj, Z. Shi, C. H. Chang, and H. Alam, “Extracting tabular information from text files,” EECS Department, Tufts University, Medford, USA , 1996
1996
Earlier work this paper cites.
T. Kieninger and A. Dengel, “The t-recs table recognition and analysis system,” in International Workshop on Document Analysis Systems . Springer, 1998, pp. 255–270
1998
Earlier work this paper cites.
J. Hu, R. S. Kashi, D. P. Lopresti, and G. Wilfong, “Medium-independent table detection,” in Document Recognition and Retrieval VII , vol. 3967. International Society for Optics and Photonics, 1999, pp. 291–302
1999
Earlier work this paper cites.
F. Cesarini, S. Marinai, L. Sarti, and G. Soda, “Trainable table location in document images,” in Object recognition supported by user interaction for service robots , vol. 3. IEEE, 2002, pp. 236–240
2002
Earlier work this paper cites.
M. Hurst, “A constraint-based approach to table structure derivation,” 2003
2003
Earlier work this paper cites.
B. Gatos, D. Danatsas, I. Pratikakis, and S. J. Perantonis, “Automatic table detection in document images,” in International Conference on Pattern Recognition and Image Analysis . Springer, 2005, pp. 609–618
2005
Earlier work this paper cites.
A. C. e Silva, “Learning rich hidden markov models in document analysis: Table location,” in 2009 10th International Conference on Document Analysis and Recognition . IEEE, 2009, pp. 843–847
2009
Earlier work this paper cites.
A. Shahab, F. Shafait, T. Kieninger, and A. Dengel, “An open approach towards the benchmarking of table structure recognition systems,” in Proceedings of the 9th IAPR International Workshop on Document Analysis Systems . ACM, 2010, pp. 113–120
2010
Earlier work this paper cites.
F. Shafait and R. Smith, “Table detection in heterogeneous documents,” in Proceedings of the 9th IAPR International Workshop on Document Analysis Systems . ACM, 2010, pp. 65–72
2010
Earlier work this paper cites.
J. Fang, X. Tao, Z. Tang, R. Qiu, and Y. Liu, “Dataset, ground-truth and performance metrics for table detection evaluation,” in 2012 10th IAPR International Workshop on Document Analysis Systems . IEEE, 2012, pp. 445–449
2012
Earlier work this paper cites.
M. Göbel, T. Hassan, E. Oro, and G. Orsi, “ICDAR 2013 table competition,” in 2013 12th International Conference on Document Analysis and Recognition . IEEE, 2013, pp. 1449–1453
2013
Earlier work this paper cites.
T. Kasar, P. Barlas, S. Adam, C. Chatelain, and T. Paquet, “Learning to detect tables in scanned document images using line information,” in 2013 12th International Conference on Document Analysis and Recognition . IEEE, 2013, pp. 1185–1189
2013
Earlier work this paper cites.

Similar

A. Jimeno Yepes and K. Verspoor, “Literature mining of genetic variants for curation: quantifying the importance of supplementary material,” Database , vol. 2014, 2014
2014
Cited alongside, same era.
M. Fan and D. S. Kim, “Table region detection on large-scale pdf files without labeled data,” CoRR, abs/1506.08891 , 2015
Original
2015
Cited alongside, same era.
S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in Advances in neural information processing systems , 2015, pp. 91–99
2015
Cited alongside, same era.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proceedings of International Conference on Learning Representations (ICLR) , 2015
2015
Cited alongside, same era.
L. Hao, L. Gao, X. Yi, and Z. Tang, “A table detection method for pdf documents based on convolutional neural networks,” in 2016 12th IAPR Workshop on Document Analysis Systems (DAS) . IEEE, 2016, pp. 287–292
2016
Cited alongside, same era.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 779–788
2016
Cited alongside, same era.
M. Pawlik and N. Augsten, “Tree edit distance: Robust and memory-efficient,” Information Systems , vol. 56, pp. 157–173, 2016
2016
Cited alongside, same era.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Cited alongside, same era.
D. He, S. Cohen, B. Price, D. Kifer, and C. L. Giles, “Multi-scale multi-task fcn for semantic page segmentation and table detection,” in 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) , vol. 1. IEEE, 2017, pp. 254–261
2017
Cited alongside, same era.
K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 2961–2969
2017
Cited alongside, same era.
S. Schreiber, S. Agne, I. Wolf, A. Dengel, and S. Ahmed, “Deepdesrt: Deep learning for detection and structure recognition of tables in document images,” in 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) , vol. 1. IEEE, 2017, pp. 1162–1167
2017
Cited alongside, same era.
A. Gilani, S. R. Qasim, I. Malik, and F. Shafait, “Table detection using deep learning,” in 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR) , vol. 1. IEEE, 2017, pp. 771–776
2017
Cited alongside, same era.

Then

N. Siegel, N. Lourie, R. Power, and W. Ammar, “Extracting scientific figures with distantly supervised neural networks,” in Proceedings of the 18th ACM/IEEE on joint conference on digital libraries . ACM, 2018, pp. 223–232
2018
Later among the works it cites.
P. W. Staar, M. Dolfi, C. Auer, and C. Bekas, “Corpus conversion service: A machine learning platform to ingest documents at scale,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . ACM, 2018, pp. 774–782
2018
Later among the works it cites.
Y. Deng, D. Rosenberg, and G. Mann, “Challenges in end-to-end neural scientific table recognition,” in 2019 International Conference on Document Analysis and Recognition (ICDAR) . IEEE, Sep. 2019, pp. 894–901
2019
Closest in time.
S. R. Qasim, H. Mahmood, and F. Shafait, “Rethinking table recognition using graph neural networks,” pp. 142–147, Sep. 2019
2019
Closest in time.
X. Zhong, J. Tang, and A. J. Yepes, “Publaynet: largest dataset ever for document layout analysis,” in 2019 International Conference on Document Analysis and Recognition (ICDAR) . IEEE, Sep. 2019, pp. 1015–1022
2019
Closest in time.
L. Gao, Y. Huang, Y. Li, Q. Yan, Y. Fang, H. Dejean, F. Kleber, and E.-M. Lang, “ICDAR 2019 competition on table detection and recognition,” in 2019 International Conference on Document Analysis and Recognition (ICDAR) . IEEE, Sep. 2019, pp. 1510–1515
2019
Closest in time.
S. S. Paliwal, D. Vishwanath, R. Rahul, M. Sharma, and L. Vig, “Tablenet: Deep learning model for end-to-end table detection and tabular data extraction from scanned document images,” in 2019 International Conference on Document Analysis and Recognition (ICDAR) . IEEE, 2019, pp. 128–133
2019
Closest in time.
I. Kavasidis, C. Pino, S. Palazzo, F. Rundo, D. Giordano, P. Messina, and C. Spampinato, “A saliency-based convolutional neural network for table and chart detection in digitized documents,” in International Conference on Image Analysis and Processing . Springer, 2019, pp. 292–302
2019
Closest in time.
C. Tensmeyer, V. I. Morariu, B. Price, S. Cohen, and T. Martinez, “Deep splitting and merging for table structure decomposition,” in 2019 International Conference on Document Analysis and Recognition (ICDAR) . IEEE, 2019, pp. 114–121
2019
Closest in time.
P. Riba, A. Dutta, L. Goldmann, A. Fornés, O. Ramos, and J. Lladós, “Table detection in invoice documents by graph neural networks,” in 2019 International Conference on Document Analysis and Recognition (ICDAR) . IEEE, Sep. 2019, pp. 122–127
2019
Closest in time.
Y.-F. Zhou, R.-H. Jiang, X. Wu, J.-Y. He, S. Weng, and Q. Peng, “Branchgan: Unsupervised mutual image-to-image transfer with a single encoder and dual decoders,” IEEE Transactions on Multimedia , 2019
2019
Closest in time.
R. Morais, V. Le, T. Tran, B. Saha, M. Mansour, and S. Venkatesh, “Learning regularity in skeleton trajectories for anomaly detection in videos,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2019, pp. 11 996–12 004
2019
Closest in time.
K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudinov, R. Zemel, and Y. Bengio, “Show, attend and tell: Neural image caption generation with visual attention,” in International conference on machine learning , 2015, pp. 2048–2057
2057
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…