Text Mining and Analysis: Topic Modeling with Network Visualization


Network visualization can be a very useful tool for text mining and analysis. If we represent a text as a network, where the words are the nodes and the co-occurrences are the relations between them, we can then use the methods from network science to discover the patterns and their relations using the graph theory. We can find out:

  1. The most influential keywords
  2. The local contexts they appear in
  3. Topic modeling: most relevant keyword clusters
  4. How they compare to the topics discovered through LDA
  5. The structure of the discourse (diverse or biased)
  6. The structural gaps (what's missing that could've been in the text)

There are many different areas where this approach can be applied: for example, in literature studies, in creative and scientific writing, marketing research and to augment various text analysis applications.

We can then use this information to get a general overview of a text or to get an insight into how it could be developed further, giving rise to the new ideas and research questions. Read more about our approach in our text network analysis research papers on Nodus Labs.


Text Mining Using a Network Graph in 4 Steps


In order to try the text mining app on your data, log in InfraNodus and choose the "Add a new text" app on the apps screen.

You will see a graph where the most prominent terms are bigger and the terms that belong to the same topic have the same color and are closer, so you can better understand the main topics of the text and its structure.

Visualize google search results as a text network graph

You can then select the most prominent keywords to see what parts of text they relate to and to better understand the context they appear in. Like an upgraded Tagcloud.

Zoom into the topic and explore it further

You can also select a topic you're interested in and explore that direction.

Select the most relevant keywords from your search to see relevant results.

You can also generate new ideas in relation to the text if you look at the structural gaps in the graph using the Insight feature.

Select the most relevant keywords from your search to see relevant results.



The short video tutorial below explains this process in detail and shows the workflow that you can follow when you want to do text mining using network visualization.



 

How Does It Work?


The basic approach is based on our paper: InfraNodus: Generating Insight Using Network Analysis (Paranyushkin, 2019).

First, we import the text that you added. Then we extract the lemmas and each unique word is represented as a node while co-occurrence of words (within a certain gap) is represented as a graph edge.

Next, we apply several node ranking and community detection measures, which range the nodes by betweenness centrality (discoursive influence) and groups them by communities (the nodes that are more densely connected together belong to the same topic, have the same color, and are located closer on the graph.



Identify the Level of Text's Diversity


You can use text network analysis to get insights about the text's structure, so you can see how connected or dispersed (diverse) it is and get a new perspective on your data.





Try It Yourself


You can try this approach yourself using InfraNodus for any search query you're interested in:


Log In    Sign Up
 

 

Custom Text Mining Using Network Analysis


The application above works best for single texts, writing work-in-progress, and articles. If you are interested to apply the same method to a big text corpora there may be some scalability issues related to saturation.

However, using our in-house tools we can help you design custom analysis of your own text corpora. Please, let us know if you have an interesting proposition, use case, research project, or service request.


Contact Us