Text Extraction from Image using Python

@ IJTSRD | Available Online @ www.ijtsrd.com
ISSN No: 2456
International
Research
Text Extraction from Image using Python
T. Gnana Prakash
Assistant Professor, CSE Department,
VNR VJIET, Hyderabad, India
ABSTRACT
With so much of our lives computerized, it is vitally
important that machines and humans can understand
one another and pass information back and forth.
Mostly computers have things their way we have to &
talk to them through relatively crude devices such
keyboards and mice so they can figure out what we
want them to do. However, when it comes to
processing more human kinds of information, like an
old-fashioned printed book or a letter scribbled with a
fountain pen, computers have to work much harder.
That is where optical character recognition (OCR)
comes in. Here we process the image, where we apply
various pre-processing techniques like desk wing,
binarization etc. and algorithms like Tesseract to
recognize the characters and give us the final
document.
Keywords: Open CV- Python; Image Processing; Text
Extraction; Image threshold; Virtual Image
I. INTRODUCTION
Text data present in images contain useful
information for automatic annotation, indexing, and
structuring of images. Extraction of this information
involves detection, localization, tracking, extraction,
enhancement, and recognition of the text from a giv
image. However, variations of text due to differences
in size, style, orientation, and alignment, as well as
low image contrast and complex background make
the problem of automatic text extraction extremely
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 1 | Issue – 6 | Sep-Oct 2017
ISSN No: 2456 - 6470 | www.ijtsrd.com | Volume
International Journal of Trend in Scientific
Research and Development (IJTSRD)
International Open Access Journal
Professor, CSE Department,
India
K. Anush
Assistant Professor, CSE Department,
Vardhaman College of Engineering, Hyderabad, I
With so much of our lives computerized, it is vitally
important that machines and humans can understand
one another and pass information back and forth.
Mostly computers have things their way we have to &
talk to them through relatively crude devices such as
keyboards and mice so they can figure out what we
want them to do. However, when it comes to
processing more human kinds of information, like an
fashioned printed book or a letter scribbled with a
fountain pen, computers have to work much harder.
at is where optical character recognition (OCR)
comes in. Here we process the image, where we apply
processing techniques like desk wing,
binarization etc. and algorithms like Tesseract to
recognize the characters and give us the final
Python; Image Processing; Text
Extraction; Image threshold; Virtual Image
Text data present in images contain useful
information for automatic annotation, indexing, and
structuring of images. Extraction of this information
involves detection, localization, tracking, extraction,
enhancement, and recognition of the text from a given
image. However, variations of text due to differences
in size, style, orientation, and alignment, as well as
low image contrast and complex background make
the problem of automatic text extraction extremely
challenging. While comprehensive surveys o
problems such as face detection, document analysis,
and image indexing can be found, the problem of text
information extraction is not well surveyed. A large
number of techniques have been proposed to address
this problem, and the purpose of this
classify and review these algorithms, discuss
benchmark data and performance evaluation, and to
point out promising directions for future research.
Content-based image indexing refers to the process of
attaching labels to images based on the
Image content can be divided into two main
categories: perceptual content and semantic content.
Perceptual content includes attributes such as color,
intensity, shape, texture, and their temporal changes,
whereas semantic content means objects,
their relations. A number of studies on the use of
relatively low-level perceptual content for image and
video indexing have already been reported. Studies on
semantic image content in the form of text, face,
vehicle, and human action have als
recent interest. Among them, text within an image is
of particular interest as
 It is very useful for describing the contents of
an image;
 It can be easily extracted compared to other
semantic contents, and
 It enables applications such
image search, automatic video logging, and
text-based image indexing.
Oct 2017 Page: 310
www.ijtsrd.com | Volume - 1 | Issue – 6
Scientific
(IJTSRD)
International Open Access Journal
Anusha
Professor, CSE Department,
ngineering, Hyderabad, India
challenging. While comprehensive surveys of related
problems such as face detection, document analysis,
and image indexing can be found, the problem of text
information extraction is not well surveyed. A large
number of techniques have been proposed to address
this problem, and the purpose of this paper is to
classify and review these algorithms, discuss
benchmark data and performance evaluation, and to
point out promising directions for future research.
based image indexing refers to the process of
attaching labels to images based on their content.
Image content can be divided into two main
categories: perceptual content and semantic content.
Perceptual content includes attributes such as color,
intensity, shape, texture, and their temporal changes,
whereas semantic content means objects, events, and
their relations. A number of studies on the use of
level perceptual content for image and
video indexing have already been reported. Studies on
semantic image content in the form of text, face,
vehicle, and human action have also attracted some
recent interest. Among them, text within an image is
t is very useful for describing the contents of
t can be easily extracted compared to other
semantic contents, and
It enables applications such as keyword-based
image search, automatic video logging, and
based image indexing.

International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456-6470
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 1 | Issue – 6 | Sep-Oct 2017 Page: 311
II. TEXT IN IMAGES
A variety of approaches to text information extraction
(TIE) from images have been proposed for specific
applications including page segmentation, address
block location, license plate location, and content-
based image indexing.
Fig. 1: Grayscale document images
Fig. 2: Multi-color document images
Fig. 3: Images with caption text
Fig. 4: Scene text images
Text in images can exhibit many variations with
respect to the properties like geometry, color, motion,
edge and compression.
Table 1: Properties of text in images
Properties Variants or sub-
classes
Geometry
Size Regularity in size of
text
Alignment
Horizontal/vertical
Straight line with
skew (implies vertical
direction)
Curves
3D perspective
distortion
Inter-
character
distance
Aggregation of
characters with
uniform distance
Colour
Gray
Colour (monochrome,
polychrome)
Motion
Static
Linear Movement
2D rigid constrained
movement
3D rigid constrained
movement
Free Movement
Edge Strong edges
(contrast) at text
boundaries
Compression
Un-compressed image
JPEG, MPEG-
compressed image
The problem of Text Information Extraction TIE
system receives an input in the form of a still image or
a sequence of images. The images can be in gray scale
or color, compressed or un-compressed, and the text
in the images may or may not move. The TIE problem
can be divided into the following sub-problems: (i)
detection, (ii) localization, (iii) tracking, (iv)
extraction and enhancement (v) Optical Character
recognition (OCR).

International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456
III. IMAGE THRESHOLDING
A. Threshold Binary
Fig.5: Threshold Binary
This thresholding operation can be expressed as:
So, if the intensity of the pixel
, then the new pixel intensity is set to a
. Otherwise, the pixels are set to
B. Threshold Binary, Inverted
Fig. 5: Threshold Binary, Inverted
If the intensity of the pixel
Otherwise, it is set to .
C. Truncate
Fig. 6: Truncate
The maximum intensity value for the pixels is
is greater, then its value is
truncated. See figure below:
, if
Fig.5: Threshold Binary
is higher than
. Otherwise, the pixels are set to .
Fig. 5: Threshold Binary, Inverted
is higher than
, then the new pixel intensity is set to a .
The maximum intensity value for the pixels is
is greater, then its value is
D. Threshold to Zero
Fig. 7: Threshold to Zero
This operation can be expressed as:
If is lower than
be set to .
E. Threshold to Zero, Inverted
Fig. 8: Threshold to Zero, Inverted
If is greater than
will be set to .
F. Simple Thresholding
If pixel value is greater than a threshold value, it is
assigned one value (may be white), else it is assigned
another value (may be black). The function used is
cv2.threshold. First argument is the source image,
which should be a grayscale image
is the threshold value which is used to classify the pixel
values. Third argument is the maxVal which represents
the value to be given if pixel value is more than
(sometimes less than) the threshold value. OpenCV
provides different styles of thresh
decided by the fourth parameter of the function.
Different types are:
 cv2.THRESH_BINARY
 cv2.THRESH_BINARY_INV
Oct 2017 Page: 312
Fig. 7: Threshold to Zero
, the new pixel value will
ero, Inverted
Fig. 8: Threshold to Zero, Inverted
, the new pixel value
If pixel value is greater than a threshold value, it is
assigned one value (may be white), else it is assigned
another value (may be black). The function used is
. First argument is the source image,
should be a grayscale image. Second argument
is the threshold value which is used to classify the pixel
values. Third argument is the maxVal which represents
the value to be given if pixel value is more than
(sometimes less than) the threshold value. OpenCV
provides different styles of thresholding and it is
decided by the fourth parameter of the function.
cv2.THRESH_BINARY_INV

 cv2.THRESH_TRUNC
 cv2.THRESH_TOZERO
 cv2.THRESH_TOZERO_INV
Two outputs are obtained. First one is a
output is our thresholded image.
Fig. 9: Image Describing outputs of different
Thresholding techniques
IV. PYTHON ANYWHERE
Python Anywhere is an online Integrated
Development Environment (IDE) and Web hosting
service based on the Python programmi
provides in browser access to server-based Python and
Bash Command-line interfaces, along with a code
editor with Syntax highlighting. One striking different
between Python Anywhere and the usual Python Cloud
Computing solution that we know of, is that you can
totally work on it online using internet browser in
developing your Python application. With this, you can
bypass the usual delicacies on preparing a local
workstation that meet cloud hosting service
environment requirement and directly work inside
your browser that connected to many consoles
provided by Python anywhere, such as : Bash,
Python/iPython 2.6/2.7/3.3 and MySQL.
This provides a step-by-step guide on how to deploy
your Django applications. The service provides in
browser access to the server-based Python and Bash
command line interfaces, meaning you can interact
with Python Anywhere’s servers just like you would
with a regular terminal instance on your own
computer. Currently, Python Anywhere are offering a
free account which sets you up with an adequate
amount of storage space and CPU time to get a Django
application up and running.
Two outputs are obtained. First one is a retval . Second
Fig. 9: Image Describing outputs of different
Thresholding techniques
is an online Integrated
Development Environment (IDE) and Web hosting
service based on the Python programming language. It
based Python and
line interfaces, along with a code
editor with Syntax highlighting. One striking different
between Python Anywhere and the usual Python Cloud
of, is that you can
totally work on it online using internet browser in
developing your Python application. With this, you can
bypass the usual delicacies on preparing a local
workstation that meet cloud hosting service
y work inside
your browser that connected to many consoles
provided by Python anywhere, such as : Bash,
Python/iPython 2.6/2.7/3.3 and MySQL.
step guide on how to deploy
your Django applications. The service provides in-
based Python and Bash
command line interfaces, meaning you can interact
with Python Anywhere’s servers just like you would
with a regular terminal instance on your own
computer. Currently, Python Anywhere are offering a
h sets you up with an adequate
amount of storage space and CPU time to get a Django
A. Creating a Python Anywhere Account
First sign up for a Beginner Python Anywhere
account. If your application takes off and becomes
popular, you can always upgrade your account at a
later stage to gain more storage space and CPU time
along with a number of other benefits (like hosting
specific domains and ssh abilities).
Once your account has been created, you will have
your own little slice of the World Wide Web at
http://<username>.pythonanywhere.com, where
<username> is your Python Anywhere username. It is
from this URL that your hosted a
available from.
B. The Python Anywhere Web Interface
The Python Anywhere web interface contains a
dashboard, which in turn provides a series of tabs
allowing you to manage your application. The tabs as
illustrated in Fig. 10 include:
 a consoles tab, allowing you to create and
interact with Python and Bash console
instances;
 a files tab, which allows you to upload to and
organize files within your disk quota;
 a web tab, allowing you to configure settings
for your hosted web applicatio
 a schedule tab, allowing you to setup tasks to
be executed at particular times; and
 a databases tab, which allows you to configure
a MySQL instance for your applications should
you require it.
Of the five tabs provided, we’ll be working primarily
with the consoles and web tabs. The Python Anywhere
help pages provide a series of detailed explanations on
how to use the other tabs.
Fig. 10: The Python Anywhere dashboard, showing
the Consoles
Oct 2017 Page: 313
A. Creating a Python Anywhere Account
First sign up for a Beginner Python Anywhere
account. If your application takes off and becomes
popular, you can always upgrade your account at a
later stage to gain more storage space and CPU time
along with a number of other benefits (like hosting
ic domains and ssh abilities).
Once your account has been created, you will have
your own little slice of the World Wide Web at
http://<username>.pythonanywhere.com, where
<username> is your Python Anywhere username. It is
from this URL that your hosted application will be
B. The Python Anywhere Web Interface
The Python Anywhere web interface contains a
, which in turn provides a series of tabs
allowing you to manage your application. The tabs as
tab, allowing you to create and
interact with Python and Bash console
tab, which allows you to upload to and
organize files within your disk quota;
tab, allowing you to configure settings
for your hosted web application;
tab, allowing you to setup tasks to
be executed at particular times; and
tab, which allows you to configure
a MySQL instance for your applications should
Of the five tabs provided, we’ll be working primarily
tabs. The Python Anywhere
provide a series of detailed explanations on
Fig. 10: The Python Anywhere dashboard, showing
Consoles tab.

C. Python Anywhere to upload the image
Fig. 11: Python Anywhere IDE to upload image
In the Python Anywhere IDE, the user can upload the
image from which he or she wishes to extract the text.
After logging into Python Anywhere account, a user
has to go to the working directory where one can find
“Upload a File” option. Clicking on it lets user chose
the desired image and then uploads it to Python
Anywhere cloud.
D. The Bash Console
Fig. 12: Finding Bash Console in Python Anywhere
Python Anywhere allows a user to have two consoles
for a free trial. On upgrading the account, a user can
increase this number. To run the python files one must
open the bash console.
Fig. 13: Running Files in Bash Console
Here we specify the file we wish to run. Python is the
keyword to specify that we are running a python file
and testest.py is the file name.
E. Result File
Fig. 14: Text files containing extracted text
The text extracted from the images is pipelined to a
text file where the user can view, edit and modify its
contents. User can thus save the obtained text file and
download it from Python Anywhere.
V. SYSTEM ANALYSIS
A. System Architecture
The entire process can be depicted using these basic
steps:
Fig. 15: Workflow in the system
The three basic steps involved in this process are
detection, enhancement and extraction. This diagram
defines the structure of the system.

Fig. 16: Detailed Architecture of system
VI. Test cases
Table 2: Test Cases
S.
No
Test Case Expected
Result
Actual Result
1 Image with
plain Text and
plain
background
Text
extracted
Text
extracted
Passed
2 Image with
luminance
Text
extracted
Text
extracted
Passed
3 Tabular data
which contains
the rows and
columns
Text
extracted
Text
extracted
Passed
4 Letter head Text
extracted
Text
extracted
Passed
5 Bond paper
with the text
content which
is in colour
Text
extracted
Text
extracted
Passed
6 Signboard
containing text
Text
extracted
Text
extracted
Passed
7 Text with
varying font
size
Text
extracted
Text
extracted
Passed
8 Handwritten
text
Text
extracted
Partiall
y
passed
9 Image with
high text data
of low details
Text
extracted
Text
extracted
Failed
10 Complex
background
image with
tilted text
containing
mixed colours
Text
extracted
Text
extracted
Failed
11 Label on water
bottle
Text
extracted
Text
extracted
Failed
Below are the results of few test cases performed. The
original image and the extracted text are shown
below.
A. Example 1:
Fig.17: Image
Fig. 18: Image with plain background
B. Example 2:
Fig. 19: Bond Paper

Fig. 20: Bond Paper with plain background
C. Example 3:
Fig. 21: Hand writing Image
Fig. 22: Image with plain background
D. Example 4:
Fig. 23: Image with high text data of low details
Fig. 24: Image with high text data of low details
E. Example 5:
Fig. 25: Complex background image with tilted text
containing mixed colors
Fig. 25: Complex background image with tilted text
containing mixed colors with plain background
CONCLUSION
Even though a large number of algorithms have been
proposed in the literature, no single method can
provide satisfactory performance in all the
applications due to the large variations in character
font, size, texture, color, etc. Through this paper we
are in the stream of deriving the satisfactory results
by enhancing the input by fine tuning the image and
deriving the optimum levels of accuracy from
TESSERACT.

FUTURE SCOPE
With machine learning algorithms constantly being
developed and improved, massive amounts of
computational power becoming readily available both
locally and on the cloud, and unfathomable amounts
of data can be extracted not only in the domain of
image but also in terms of scene, video frames and
scrolling types of data.
REFERENCES
1) M. Flickner, H. Sawney et al., Query by
Image and Video Content: The QBIC System,
IEEE Computer 28 (9) (1995) 23-32.
2) J. Zhang, Y. Gong, S. W. Smoliar, and S. Y.
Tan, Automatic Parsing of News Video, Proc.
of IEEE Conference on Multimedia
Computing and Systems, 1994, pp. 45-54.
3) M. H. Yang, D. J. Kriegman, and N. Ahuja,
Detecting faces in Images: A Survey, IEEE
Transactions on Pattern Analysis and Machine
Intelligence, 24 (1) (2002) 34-58.
4) Y. Cui and Q. Huang, Character Extraction of
License Plates from Video, Proc. of IEEE
Conference on Computer Vision and Pattern
Recognition, 1997, pp. 502 –507.
5) C. Colombo, A. D. Bimbo, and P. Pala,
Semantics in Visual Information Retrieval,
IEEE Multimedia, 6 (3) (1999) 38-53.
6) T. Sato, T. Kanade, E. K. Hughes, and M. A.
Smith, Video OCR for Digital News Archive,
Proc. of IEEE Workshop on Content based
Access of Image and Video Databases, 1998,
pp. 52-60.
7) Atsuo Yoshitaka and Tadao Ichikawa, A
Survey on Content-based Retrieval for
Multimedia Databases, IEEE Transactions on
Knowledge and Data Engineering, 11(1999)
81-93.
8) W. Qi, L. Gu, H. Jiang, X. Chen, and H.
Zhang, Integrating Visual, Audio, and Text
Analysis for News Video, Proc. of IEEE
International Conference on Image
Processing, 2000, pp. 10-13.
9) D.Wactlar, T. Kanade, M. A. Smith, and S. M.
Stevens, Intelligent Access to Digital Video:
The Informedia Project, IEEE Computer, 29
(5) (1996) 46-52.
10) H. Rein-Lien, M. Abdel-Mottaleb, A. K. Jain,
Face Detection in Color Images, IEEE
Transactions on Pattern Analysis and Machine
Intelligence, 24 (5) (2002) 696-706.

Text Extraction from Image using Python

Recommended

More Related Content

What's hot (20)

Similar to Text Extraction from Image using Python (20)

More from ijtsrd (20)

Recently uploaded (20)

Text Extraction from Image using Python