How Search Queries Trigger Structured Information Cards (Knowledge Panels)

How might queries that trigger structured information cards change over time?

When does a search engine decide that it should show a knowledge panel in response to a query?

What words in a query will trigger that knowledge panel?

A knowledge panel is sometimes referred to as a structured information card by Google.

Other structured information cards contain information about things such as hotel reservations, flight arrivals, dinner reservations, movie tickets, and others.

If you’ve worked with knowledge panels, you’ve learned that different searches will trigger those to appear.

Often those include a mention of an entity, such as a business name, or a certain thing.

And queries that trigger structured information cards can change over time according to a recently granted patent.

Search queries can be used to return resources – such as web pages, images, text documents, electronic mail documents, multimedia content, etc. – relevant to a searcher’s needs and to present information about those resources in a way that is most useful to the searcher.

Sometimes the best result may be a structured information card.

A computer system may:

• Receive a search query.
• Process the search query.
• Identify results that are relevant to the search query.
• Return a set of search results in response to a searcher submitted query.

The patent this post is about is from the start of November 2020, and it tells us about a card trigger-term identification unit that could identify additional trigger-terms that show a structured information card.

The card trigger-term identification unit allows the grammar of one or more structured information cards to be tuned, over time, by evaluating candidate terms in queries for potential inclusion in the grammar of a structured information card.

For example, assume the grammar for a “Movie” structured information card includes terms such as “movie time,” “movie ticket confirmation,” and “ticket confirmation number.”

The card trigger-term identification unit may:

• Analyze terms associated with the grammar of the “Movie” structured information card and one or candidate queries.
• Identify an additional trigger-term for the “Movie” structured information card such as the trigger-term “movie ticket.”

Accordingly, follow-up queries that are received may include terms such as “movie time”, “movie ticket,” or both and will trigger the display of a “Movie” structured information card in response to such queries.

The subject matter of this patent may be used to identify additional trigger-terms that will show structured information cards.

The process behind the patent may include accessing data associated with a template for presenting structured information in response to a search query, wherein the accessed data references:

1. One or more label terms that, when included in the search query, triggers a structured information card to be presented according to the template.
2. For each of the one or more label terms, a value, obtaining a candidate label term that is not already associated with the template for presenting the structured information.

For each of the one or more label terms:

• Identifying entities that are associated with the label term.
• Identifying entities that are associated with the candidate label term.

For each of the entities associated with a candidate label term, a query may cause an association, with a candidate label term:

• One or more of the label terms that are associated with the entity.
• For each of the one or more of the label terms that are associated with the entity, the value associated with the label term, and after receiving a query that includes the candidate label term.
• Using the one or more values associated with the candidate label term to determine whether to trigger the structured information to be presented according to the template.

These and other versions may optionally include one or more of the following features:

• The label terms may correspond to parameters of a search query.
• The value may be indicative of the number of times the query has been used to trigger the appearance of the structured information card.
• Obtaining a candidate label term that is not already associated with the template for presenting the structured information card may involve identifying query terms from a query log.
• Using the one or more values associated with the candidate label term to determine whether to trigger the structured information to be presented according to the template may include aggregating the one or more values that are associated with the candidate label term.
• Determining whether the aggregated value satisfies a predetermined threshold, and in response to determining that the aggregated value satisfies the predetermined threshold.
• Determining that the search query including the candidate label term will trigger the presentation of the structured information.

Triggering Structured Information Cards

Using the values associated with a candidate label term to trigger a structured information card to be presented according to the template may involve:

• Aggregating the one or more values that are associated with the candidate label term.
• Determining whether the aggregated value satisfies a predetermined threshold.
• In response to determining that the aggregated value exceeds the predetermined threshold, determining that the search query including the candidate label term will not trigger the presentation of the structured information.

The method behind the patent may also include adjusting the values that are associated with candidate labels based on those candidate label’s similarity to the label terms.

This structured information cards patent can be found at:

Search and retrieval of structured information cards

Inventors: Marc Alexander Najork, Sujith Ravi, Michael Bendersky, Peter Shao-sen Young, Timothy Youngjin Sohn, Mingyang Zhang, Thomas Nelson, and Xuanhui Wang
Assignee: Google LLC
US Patent: 10,824,630
Granted: November 3, 2020
Filed: October 26, 2016

Abstract

Methods, systems, apparatus, including computer programs encoded on a computer storage medium, to facilitate identification of additional trigger-terms for a structured information card. In one aspect, the method includes actions of accessing data associated with a template for presenting structured information, wherein the accessed data references:

(i) a label term and
(ii) a value.

Other actions may include obtaining a candidate label term, identifying one or more entities that are associated with the label term, identifying one or more of the entities that are associated with the candidate label term, and for each particular entity of the one or more entities that are associated with the candidate label term, associating, with the candidate label term, (i) a label term that is associated with the particular entity, and (ii) the value associated with the label term.

A System for Identifying Additional Trigger – Terms for a Structured Information Card

The patent shows off some examples of information that might be used to create structured information cards in a drawing:

It provides an example of a structured information card involving flight information.

Assume a searcher is standing at an airport ticket counter before his or her flight #437 to Denver, Colorado.

To check their bags, and obtain a boarding pass, they need to provide a flight confirmation number for their upcoming flight.

To respond to a request for a flight confirmation number from an airline employee, the searcher may look for a confirmation email from the airline that includes their flight confirmation number.

To obtain that confirmation email, the searcher may search for a query that includes the term “Flight Ticket” into a search box at the search engine.

After receiving a query, the search device may send the query to a server using a network.

The server may process the query, identify search results responsive to the query term “Flight Ticket,” and then return the search results to the search device.

The search results may be received by the search device and provided for display using the interface of the search device.

The search results may include references to email documents. The references may include a link that, when selected, provides an email document associated with the link to display on the search device.

Each respective email reference may include text such as the name of the email sender, the subject line of the email, the time the email was received, and the date the email was received.

The search results identified based on the query “Flight Ticket” may include emails from a variety of different senders.

The search results may refer to an email from a movie theater “ABC Theatre” related to the searcher’s purchase of a movie ticket to see “Pilot’s First Flight.”

The search results may also refer to an email from a restaurant reservation service “Closed Table” for a reservation at the “Wine Flight Bar.”

The search results may also refer to an email of an order confirmation from “DC Outfitters” for the purchase of a “Flight Jacket.”

The search results may refer to emails associated with airline ticket purchase confirmations from an airline such as “NE Airlines,” “SE Airlines,” or the like.

One particular reference is for a confirmation email that the searcher received after purchasing an airline ticket from “NE Airlines” for “Flight 437.”

However, given the number of search results provided in response to the search query, it could be time-consuming to obtain the searcher’s flight confirmation number.

This is because the searcher must read the sender and subject line of each reference to each respective email that is responsive to the search.

It may not be until the searcher reads the sender and subject line of each email that the searcher will even identify the correct email that includes the flight confirmation number.

Then, the searcher using the search device must request the email document associated with the reference.

Finally, after the email document is selected, the email document with the flight confirmation number for the searcher’s upcoming Flight #437 is returned.

Though the searcher was able to obtain the user’s confirmation number, the process was inefficient.

In addition, the process may not go smoothly for reasons below and the like, such as:

• Pressure on the user because of a long line of passengers at the ticket counter.
• Slow cellular data signals due to poor reception in the airport terminal.

This system may work to improve the searcher’s retrieval of information from the searcher’s email inbox.

For instance, at stage A, the system provides multiple structured information cards stored in a structured information card storage unit.

Each structured information card includes a template and a grammar.

The template of the structured information card includes one or more predetermined fields that can be populated with information from an email document in response to a particular search query.

For example, the structured information storage unit may include a structured information card for “Flight” information.

The structured information card for “Flight” information may include:

• A “To” field
• A “From” field
• A “Departs” field
• A “Conf. #” field
• A “Flight #” field

Trigger Terms Associated with Structured Information Cards

The grammar of structured information cards may include one or more trigger-terms that are associated with the card.

A trigger-term may include terms that, when detected by the server as being included in a search query, trigger the retrieval, population, and the display of the structured information card associated with the trigger-term.

So, in response to a query that includes the term “flight reservation,” the “Flight” structured information card may be:

• Obtained.
• Populated with data from the most recent email that is associated with an upcoming flight.
• Provided for display via a searcher interface.

However, the “Flight” structured information card may not have been triggered in response to a query including the term “Flight Ticket” because the search term “Flight Ticket” may not have been included in the “Flight” structured information card’s grammar at an initial stage A.

The search server may use the card trigger-term identification unit to identify additional terms that trigger the “Flight” structured information card.

Those additional terms may be based on queries received from a search box associated with a user interface displayed by a search device such as a search box.

The card trigger-term identification unit could:

• Obtain a query term from a log of received query terms.
• Determine if the query term is related to one or more other terms in a structured information card’s grammar.
• Add the query term to the structured information card’s grammar.

The card trigger-term identification unit may determine that the term “Flight Ticket” should be added to the grammar of the “Flight” structured information card.

Adding the term “Flight Ticket” to the grammar of the “Flight” structured information card results would be done in an updated structured information storage unit at stage B of the process.

At stage B, the searcher of the same search device can access a user interface at a later point in time.

The search interface may be the same searcher interface. The searcher may input a search query that includes the term “Flight Ticket” into the search box.

The search device may transmit the search query to a server.

The server may process the query, identify search results responsive to the search query “Flight Ticket,” and then return the search results to the search device.

The search results may be received by the search device and provided for display using the interface of the search device.

The search results may refer to email documents.

At stage B the search interface includes a structured information card.

That structured information card may include a display with fields that are populated with data extracted from a resource that is responsive to the search query.

The structured information card may be obtained, populated, and provided for display through the search interface because the grammar of the structured information card now includes the term “Flight Ticket.”

The term “Flight Ticket” may have been added to the grammar of the “Flight” structured information card based on:

• The card trigger-term identification unit’s analysis of the existing terms included within the grammar of the structured information card.
• Identified relationships between existing terms included within the grammar of the structured information card.
• Identified relationships between the aforementioned grammar terms and one or more queries previously submitted via the search box.

The particular structured information card obtained and displayed may be based on query terms submitted through the search box.

For example, the server may select a particular structured information card for display via the user interface based on a determination that the search query term such as “Flight Ticket” matches one or more grammar terms associated with the particular structured information card.

The server may populate the obtained structured information card with the contents from the highest-ranked search result that includes information requested by the fields of the structured information card template.

Those highest-ranked search results might be from the most recent email document that includes information requested by the fields of the structured information card template.

Advantages of This Structured Information Card Approach

It can display relevant information related to the searcher’s upcoming flight without requiring the searcher to read the data associated with each reference returned as a search result by the server.

The structured information card displays:

• The user’s flight destination (e.g., Denver, Co).
• The user’s flight origin (e.g., Washington, D.C.).
• The user’s flight departure time (e.g., 11:45 a.m. EST).
• The user’s flight confirmation number (e.g., KP4EG).
• The user’s flight number (e.g., 437).

The searcher does not need to open the email including information about the searcher’s upcoming flight because the necessary information associated with the user’s upcoming flight is within the structured information card.

Because of this, the searcher using the search device who is standing at the ticket counter can:

• Quickly search their email.
• Obtain their flight confirmation number from the structured information card.
• Provide the flight confirmation number to the airline representative in an efficient manner.

The patent description provides an example of a system that uses a card trigger-term identification unit to identify additional terms that can be added to the grammar of a structured information card is directed towards a “Flight” structured information card.

However, the description of the patent tells us that it should not be so limited.

The card trigger-term identification unit can be used to identify additional grammar terms for any type of structured information card such as:

• Movie Ticket structured information cards.
• Dinner Reservation structured information cards.
• Hotel Reservation structured information cards.
• Vehicle Rental structured information cards.
• Device Rental structured information cards.
• The like.

The patent also tells us that any type of structured information card may be used where the structured information card can be uniquely identified using a set of one or more grammar terms.

Entities, Attributes, & Graph Structure Information Cards

Google identifies entities and associates labels and attributes with those entities, and described in the following flowchart drawing from the patent:

A card trigger-term identification unit analyzes existing terms associated with the grammar of one or more structured information cards.

The terms associated with the grammar of one or more structured information cards include terms that, when received in a query, trigger the display of a particular structured information card.

Analyzing existing terms associated with the grammar of one or more information cards may include the generation of a graph structure.

This graph structure may include query nodes each associated with a particular grammar term that triggers the selection, population, and display of a particular structured information card.

Each query node may be associated with a respective label term.

One query node may be associated with the label term “Flight Reservation” and another query node may be associated with the label term “Ticket.”

Label terms used to build the graph may be obtained from the structured information card storage unit, a query log, or the like.

The graph structure may also include one or more entity nodes.

The entity nodes may include an item of data that is indicative of a relationship between the respective label terms of one or more nodes.

The relationship may include a semantic relationship associated with the label terms.

By way of example, the card trigger-term identification unit may obtain the candidate query term “Flight Ticket” to evaluate the candidate query term “Flight Ticket” for potential inclusion in the grammar associated with a structured information card such the “Flight” structured information card.

The query term “Flight Ticket” may have been stored in a query log after the user of a user device such as user device submitted the query “Flight Ticket” to search one or more emails using an interface for an electronic mailbox such as interface before the inclusion of the term “Flight Ticket” in the grammar of the “Flight” structured information card.

A query node may be generated in the graph structure based on the candidate query term “Flight Ticket.”

The candidate query node is associated with a candidate label term “Flight Ticket.”

The information from the structured information card may be information that is in a graph structure.

For instance, when an information card is about flight information, it contains key/value pairs that provide information about related entities and attributes of those entities (making it structured information).

Referring to it as a card means that it is using a display format with a related template for that format.

For flight information, you would have a departure city, a destination city, a departure time, and an arrival time, a departure airport and an arrival airport, a confirmation number, a flight number, and so on.

These related entities and attributes for them can be found in a template that has labels for each of the fields of information that it covers, and those labels can be used in a query to show an information card about a flight ticket.

They can be used for identifying an additional trigger-term for a structured information card.

This drawing from the patent shows how labels might be connected to entities and attributes:

Generally, the process may include:

• Accessing data associated with a template for presenting structured information.
• Identifying the first set of one or more entities.
• Associating one or more labels and one or more values with one or more entities in the first set of the identified entities.
• Obtaining a candidate label term.
• Identifying a subset of one or more entities from the first set of entities.
• Associating one or more labels and one or more values with one or more candidate label terms
• Receiving a search query.
• Using values associated with each candidate label term to determine whether to trigger display of the structured information.

So these candidate labels may be chosen, when they appear in queries to display a structured information card.

For example, where an entity includes an email document, network address, URL, or the like, an entity may be associated with a candidate label term if the candidate label term would return the email document, network address, URL, or the like when a query that includes the candidate label term is executed.

The system may associate one or more labels and one or more values with each candidate label term.

For example, any label terms associated with a particular entity at stage may be associated with a candidate label term with which the entity is related.

Thus a label term that was propagated to a particular entity from a query node may be further propagated from the entity to a candidate label term with which the particular entity is related.

One or more values associated with an entity may similarly be associated with one or more candidate label terms with which the entity is related.

Accordingly, a value that was propagated to a particular entity from a query node may be further propagated to a candidate label term.

This system may analyze each of the one or more values that are associated with a candidate label term to determine whether the candidate label term should be added to the grammar associated with a structured information card.

Determining whether a label term should be associated with a structured information card may include aggregating the values associated with the candidate label term, and evaluating the aggregated value against a predetermined threshold.

If it is determined that the aggregated value satisfies a predetermined threshold, the label term may be added to the grammar of the structured information card.

If the aggregated value does not satisfy a predetermined threshold, the label term is not added to the grammar of the structured information card.

Triggering a Structured Information Card

The system may process the received search query, and use values associated with each identified candidate label term to determine whether to trigger the display of a structured information card.

Using the values associated with each candidate label term may include:

• Aggregating the values associated with the candidate label term.
• Evaluating the aggregated value against a predetermined threshold.
• Determining that the aggregated value associated with the candidate label term satisfies the predetermined threshold, the system may let the search query including the candidate label term trigger a related structured information card.

Triggering Structured Information Cards Takeaways

A structured information card may appear in response to a query that is related to the grammar from the template for the different types of structured information cards.

It may be possible to anticipate which entities may be relevant for a structured information card, and which queries might trigger that card.

Structured information cards evolve in how they are triggered based on queries and the grammar of the information in the card.

More Resources:

---

Image Credits

All screenshots taken by author, December 2020