Competitive Intelligence Analysis using Data Visualization and AI

Posted Monday, March 13, 2023

Competitive intelligence is an important aspect of any marketing strategy. Using a combination of text network analysis, data visualization, and AI, it is possible to quickly understand the current market discourse in order to identify the main trends but also — what's missing.

In this tutorial, we will demonstrate how this can be done using the InfraNodus AI-powered market research functionality.

Competitive intelligence: identifying and visualization the structural gaps

The first step that many of us do when we want to understand a certain context or market is to google it. As it's 2023, we might also ask some questions to ChatGPT / GPT-4. You might also scan the websites of your competition, corporate reports, and research papers on the subject. If you are an advanced internet user, you might look into search intent using SEO tools.

Very quickly, it's too much information and it becomes difficult to see patterns, trends, and connections. That's where advanced tools from data science, network analysis, and NLP can be very useful, especially if they are used with AI-based language generation models that can be used to produce actionable insights and innovative ideas.

Using InfraNodus, you can study all those separate discourses together, revealing the main trends and keywords that the market is currently focused on. Then, you can cluster this data on a network graph and identify the structural gaps: connections between ideas that everyone else is missing (see the image above). This is where new business opportunities hide and you can use this insight to develop business strategies and innovative ideas for this particular market.

Step 1: Embed into the Current Market Discourse

The best way to begin is to use the "Contextual Supply" app in InfraNodus, which imports Google search results for a certain query and visualizes the keywords and concepts that are often used in the same context. These are shown in clusters, so you can quickly see what the current informational supply for your query is and what the market players / competitors are talking about:

Google search results for competitive intelligence data visualization

For instance, in the example above for "competitive intelligence", we can see that the market is currently supplying information on the following high-level ideas:

• 1. data gathering
• 2. market research
• 3. information analysis
• 4. deep understanding

If we want to embed our products or services into this discourse, we would somehow need to relate to those topics but to also develop them further: either by bringing in novel ideas or connecting the existing ones in a new way.

In fact, InfraNodus has a built-in summarization engine based on AI that can generate a quick pitch for this market for you. But we always recommend that you do it yourself simply by looking at the clusters on the concept map and trying to connect them in a new way. For example, in the case above, we understand that our foray into "competitive intelligence" should include something on "gathering data and information" (e.g. multiple import sources, working with data providers), how we can "analyze and review this data" better than our competition (advanced GPT based AI data analysis tools), and how this analysis can be used for "market research and studying competition".

Now that we created a summary for this market, we need to add the secret sauce that will make our offer stand out from the rest. We will do this by revealing the structural gap in the discourse: how to connect these ideas in a way that reflects the current needs of the customers. But before we do that, let's enhance our analysis with some more precise data on our competition.

Step 2: Explore Your Competitors' Discourse

To enhance the results obtained above, we can use an approach that is well-known in the context of SEO (search engine optimization). It provides really good results because you can very quickly identify what your competitors are talking about, what language they use, and — most importantly — what they are missing.

To perform this type of analysis, you need to gather the website addresses for a set of companies (for instance, the top 20 in your field). For this example, we used 20 small consulting companies. Then you can feed these URLs into the InfraNodus website analysis app, which will extract the text from those websites and visualize it as a graph. It will then use powerful NLP and AI algorithms to detect high-level ideas present in their discourse.

In our case, we identify that our direct competitors are talking about "innovation", "human centric design", and "digital transformation". So we might want to include those ideas in our pitch. Therefore, we will not only talk about our technology (e.g. "data gatherhing and analysis for market research"), but also — how it's going to forge innovation, digital transformation, and have a positive impact on all the stakeholders involved in the project.

You can then use the Structural Gap feature to detect which of those ideas could be better connected: choosing the two topics that are present in the majority of the discourses that are not connected yet. If you create content or a business idea linking those topics, you might come up with something innovative that nobody has thought of before.

Step 3: Structural Gap Analysis — Find What's Missing

Once we get an overview of the high-level ideas and topics present within a market discourse, we can use network visualization to identify the structural gaps between them and connect those ideas in a new way.

In order to do that, we can combine the two graphs above: Google search results for our query (representing the informational supply) and competition website analysis (representing the current discourse used by our competition). The resulting graph is visualized, and we can use the "Structural Gap" feature that shows which two groups of ideas could be better connected:

In this case, it's "Data gathering" and "Business strategy". You can then use the built-in AI to generate a research question or an idea that would connect those two clusters of ideas together. In the example above, AI is proposing to think of the "data gathering strategies" on competition that would improve our own business strategies. This is an interesting thought, as it is proposing us to think of how competitive intelligence can be used not only to improve our market standing, but also our internal processes and help us better compete with other organizations.

While you can use AI to generate research questions (and ideas), we always recommend using the graph to generate those questions yourself with the connections revealed in the text network. While the current AI models are powerful, nothing compares to the amount of insight you can get from just looking at data visualization and discovering some interesting links between ideas you haven't noticed before.

Step 4: Generate Innovative Ideas using GPT 3 AI

We can also use discourse visualization to generate innovative ideas for the market based on our competitive intelligence analysis data.

In order to do that, we can use the "Explore a Topic" ("GPT 4 Analysis") app in InfraNodus that uses GPT 3 AI to generate statements in relation to any query. As GPT AI generates content based on what already exists, it's a great way to get the gist of the discourse on a certain topic. We can then use this generated discourse to generate innovative ideas that can develop it further in an interesting way.

For example, here's what GPT-4 AI generates if we ask it to provide content on "market research" (note, you can just use the query directly, you do not need instructional prompts for this):

As we can see, there is quite a lot of information about "products" and "customers". If we click on those terms, we can see in which context they are used.

Once we identify a few interesting high-level idea categories, we can use AI to generate innovative ideas that would connect those topics in an interesting way. For example, linking "market research" and "product development":

The resulting idea proposes to think of a service that would use gathered customer data to design an optimal pricing strategy.

As you can see, this is much less generic than the standard AI content, because we explicitly asked the AI to connect the topics that are relevant to the main subject but in a completely new way.

Try It Yourself

Try this approach yourself using InfraNodus and your own data:

Related Tutorials

InfraNodus