Text Mining and Analysis: Topic Modeling with Network Visualization
Network visualization can be a very useful tool for text mining, data analysis and topic modeling. If we represent a text as a network, where the words are the nodes and the co-occurrences are the relations between them, we can then use the methods from network science to discover the patterns and get insights about the text's structure using the graph theory. We can find out:
- The most influential keywords
- The local contexts they appear in
- Topic modeling: most relevant keyword clusters
- How they compare to the topics discovered through LDA
- The structure of the discourse (diverse or biased)
- The structural gaps (what's missing that could've been in the text)
There are many different areas where this approach can be applied: for example, in literature studies, in creative and scientific writing, marketing research and to augment various text analysis applications.
We can then use this information to get a general overview of a text or to get an insight into how it could be developed further, giving rise to the new ideas and research questions. Read more about our approach in our text network analysis research papers on Nodus Labs.
Text Mining Using a Network Graph in 4 Steps
In order to try the text mining app on your data, log in InfraNodus and choose the "Add a new text" app on the apps screen.
First, write a 100-word abstract, copy and paste a text, or import it from a file:
You will see a graph where the most prominent terms are bigger and the terms that belong to the same topic have the same color and are closer, so you can better understand the main topics of the text and its structure.
You can then select the most prominent keywords to see what parts of text they relate to and to better understand the context they appear in. Like an upgraded Tagcloud.
You can also select a topic you're interested in and explore that direction.
You can also generate new ideas in relation to the text if you look at the structural gaps in the graph using the Insight feature.
The short video tutorial below explains this process in detail and shows the workflow that you can follow when you want to do text mining using network visualization.
Try It Yourself
You can try this approach yourself using InfraNodus on your own text data. You can use the live text-to-graph editor, import your CSV, PDF, text files, or use multiple external sources.
Log In Sign Up
Our Methodology of Text Network Analysis
The basic approach is based on our paper: InfraNodus: Generating Insight Using Network Analysis (Paranyushkin, 2019).
First, we import the text that you added. Then we extract the lemmas and each unique word is represented as a node while co-occurrence of words (within a certain gap) is represented as a graph edge.
Next, we apply several node ranking and community detection measures, which range the nodes by betweenness centrality (discoursive influence) and groups them by communities (the nodes that are more densely connected together belong to the same topic, have the same color, and are located closer on the graph.
Based on these insights obtained using network theory we perform topic modeling and extract the most influential terms. We can also use the insights we get about the text's graph structure to measure the discourse's diversity and its current trend.
Identify the Level of Text's Diversity
You can use text network analysis to get insights about the text's structure, so you can see how connected or dispersed (diverse) it is and get a new perspective on your data.
Custom Text Mining Using Network Analysis
The application above works best for single texts, writing work-in-progress, and articles. If you are interested to apply the same method to a big text corpora there may be some scalability issues related to saturation.
However, using our in-house tools we can help you design custom analysis of your own text corpora. Please, let us know if you have an interesting proposition, use case, research project, or service request.
Contact Us