Candidate brands are identified for an ad clip of a query video advertisement using two different techniques. Optical character recognition (OCR) is performed on a thumbnail of a plurality of different video ads in a curated database that includes brand-identifying metadata. In one technique, text vectorization is performed for each brand using all of the text that is extracted from each of the thumbnails for each brand, thereby providing a numerical brand embedding for each brand, a similar text vectorization is performed on the query video ad, and then a comparison is made between the embeddings to identify highest-ranking candidate brands based on brand-level identification. In another technique, a transformer-based model generates numerical embeddings for each ad clip in the curated database, and a comparison is made to these clip-based numerical embeddings to identify highest-ranking candidate brands based on clip-level identification. The results of the two techniques are combined together to provide one or more overall highest-ranking candidate brands.
Automated methods are provided for identifying when a first advertisement (ad) is a likely match of either a second ad, or one or more variants of the second ad. The first ad, the second ad, and the one or more variants of the second ad each include a plurality of sequential segments of a predefined time length, wherein the second ad and the one or more variants of the second ad are each reference ads, and the first ad is a sample ad. Vectors of segment hits are created for the various ads and are compared to each other to identify matches that represent such variants.
Systems and methods are provided for selectively replacing commercials that are in a video data stream with alternative commercials. Automatic content recognition (ACR) is performed on the video data stream to detect the identity of each of the commercials played in a commercial block during a commercial break. Commercials in the video data stream which are detected as being displayed on a video display device are stored in a first database, and commercials in the video data stream which are detected as not being displayed on the video display device are stored in a second database. A rules engine defines how commercials in the video data stream should be replaced with other commercials. Commercials in the video data stream that were previously detected as being displayed on the video display device are selectively replaced with commercials that were previously detected as not being displayed on the video display device.
H04N 21/234 - Processing of video elementary streams, e.g. splicing of video streams or manipulating MPEG-4 scene graphs
H04N 21/433 - Content storage operation, e.g. storage operation in response to a pause request or caching operations
H04N 21/442 - Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed or the storage space available from the internal hard disk
H04N 21/454 - Content filtering, e.g. blocking advertisements
4.
System and method for identifying potential commercial breaks in a video data stream by detecting absence of identified persons associated with program type content in the video data stream
Ad break boundaries, which identify start and end points of a commercial break, are identified by detecting the absence of voices and/or facial images of one or more primary cast members of a currently scheduled program.
H04N 21/439 - Processing of audio elementary streams
H04N 21/442 - Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed or the storage space available from the internal hard disk
H04N 21/422 - Input-only peripherals, e.g. global positioning system [GPS]
H04N 21/845 - Structuring of content, e.g. decomposing content into time segments
5.
Automoted identification of product or brand-related metadata candidates for a commercial using temporal position of product or brand-related text or objects, or the temporal position and audio, in video frames of the commercial
A method and system are provided for assigning metadata candidates to a commercial by performing image analysis on a plurality of the video frames to identify video frames that include one or more of identifiable product-related logos, brand-related logos, product-related text, or brand-related text which appear in the video frames, capturing frame data for such video frames, categorizing a temporal position of the identified video frames within the commercial as being in either a beginning or ending portion of the commercial, or a middle portion of the commercial, and assigning to the commercial the identified products or brands as primary metadata candidates when the temporal position of the identified video frames within the commercial is in either the beginning or ending portion of the commercial.
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
G06F 13/00 - Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
H04N 5/445 - Receiver circuitry for displaying additional information
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
Automated methods are provided for identifying when a first advertisement (ad) is a likely variant of a second ad. The first and second ads each include a plurality of sequential segments of a predefined time length, wherein the second ad is a reference ad, and the first ad is a sample ad. In one embodiment, a vector of segment hits is created for the second ad, wherein the second ad is used as both the sample ad and the reference ad for creating the vector of segment hits for the second ad. A vector of segment hits is also created for the first ad, wherein the first ad is used as the sample ad and the second ad is used as the reference ad for creating the vector of segment hits for the first ad. The vector of segment hits for the first ad is compared with the vector of segment hits for the second ad, and the first ad is identified as being a likely variant of the second ad when the vector of segment hits for the first ad matches at least a predetermined percentage of the segments of the second ad, and the vector of segment hits for the first ad is not the same as the vector of segment hits of the second ad. Other embodiments create different vectors of segment hits for the first and second ads which are likewise compared to one another to determine when ad variants are present.
An automated method is provided for identifying candidate locations in video content for inserting advertisement (ad) breaks. Each candidate location is a different offset time from the beginning of the video content. Different distinct characteristics of the video content are identified at offset times. Certain characteristics are desirable and certain other characteristics are not desirable. Candidate locations are identified which have the most desirable characteristics at particular offset times, but which do not have any of the undesirable characteristics at any of the offset times.
Black borders are detected in an image frame using a grey scale image of the image frame, and an edge image of the image frame. Candidate black borders are identified using maximum grey scale values associated with rows and columns of pixels of the grey scale image of the image frame, and then validated using a sum of grey scale values associated with rows and columns of pixels in the edge image of the image frame. If the validation fails, it is presumed that no black border exists.
Automated method and apparatus for identifying whether a first video is a content-identical variant of a second video when image frames of either the first video or the second video include at least one black border
An automated method and apparatus are provided for identifying when a first video is a content-identical variant of a second video. The first and second video each include a plurality of image frames, and the image frames of either the first video or the second video include at least one black border. A plurality of variants are generated of selected image frames of the first video and the second video. The variants are then compared to each other, and the first video is identified as being a variant of the second video when at least one match is detected among the variants.
G06V 20/40 - Scenes; Scene-specific elements in video content
10.
Consolidation of channel identifiers in electronic program guide (EPG) data for one or more EPG data providers, and use of consolidated channel identifiers for processing audience measurement data
Channel identifiers in electronic program guide (EPG) data are consolidated for one or more EPG providers, and the consolidated channel identifiers are used to automatically process audience measurement data.
H04N 21/482 - End-user interface for program selection
H04N 21/235 - Processing of additional data, e.g. scrambling of additional data or processing content descriptors
H04N 21/258 - Client or end-user data management, e.g. managing client capabilities, user preferences or demographics or processing of multiple end-users preferences to derive collaborative data
H04N 21/262 - Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission or generating play-lists
11.
Use of mobile device to provide product recommendations for an e-commerce shopping site
Recommendations are made to an e-shopper based on commercials that the e-shopper may have recently viewed on TV or heard on radio. In one preferred embodiment, a cloud-based server collects and stores TV viewership data through mobile devices, using automated content recognition (ACR).
Systems and methods are provided for selectively replacing commercials that are in a video data stream with alternative commercials. Automatic content recognition (ACR) is performed on the video data stream to detect the identity of each of the commercials played in a commercial block during a commercial break. Commercials in the video data stream which are detected as being displayed on a video display device are stored in a first database, and commercials in the video data stream which are detected as not being displayed on the video display device are stored in a second database. A rules engine defines how commercials in the video data stream should be replaced with other commercials. Commercials in the video data stream that were previously detected as being displayed on the video display device are selectively replaced with commercials that were previously detected as not being displayed on the video display device.
H04N 21/234 - Processing of video elementary streams, e.g. splicing of video streams or manipulating MPEG-4 scene graphs
H04N 21/433 - Content storage operation, e.g. storage operation in response to a pause request or caching operations
H04N 21/442 - Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed or the storage space available from the internal hard disk
H04N 21/454 - Content filtering, e.g. blocking advertisements
13.
TEXT INDEPENDENT SPEAKER-VERIFICATION ON A MEDIA OPERATING SYSTEM USING DEEP LEARNING ON RAW WAVEFORMS
An artificial neural network architecture is provided for processing raw audio waveforms to create speaker representations that are used for text-independent speaker verification and recognition. The artificial neural network architecture includes a strided convolution layer, first and second sequentially connected residual blocks, a transformer layer, and a final fully connected (FC) layer. The strided convolution layer is configured to receive raw audio waveforms from a speaker. The first and the second residual blocks both include multiple convolutional and max pooling layers. The transformer layer is configured to aggregate frame level embeddings to an utterance level embedding. The output of the FC layer creates a speaker representation for the speaker whose raw audio waveforms were inputted into the strided convolution layer.
A computer program product is provided for predicting ad spend for a specific media program aired or streamed on a specific network at a specific date and time using a database of media program data that includes known ad spend for a subset of media programs, and viewership data for each of the media programs, including total viewership and viewership ratings. Each of the media programs is identified by its respective network, and date and time of airing or streaming. A random forest model is trained to predict ad spend using variables that are identified as being correlated with ad spend. The trained random forest model is then used to predict ad spend for a specific media program that is aired or streamed on a specific network at a specific date and time, and which has an unknown ad spend.
A method is provided to identify whether video content, which includes a plurality of image frames, is likely to include an advertisement. The video content is split into a plurality of segments, each segment having a pre-specified duration. Subtitle text information is extracted from each segment and is passed through a natural language processing (NLP) language model to extract an embedding representing the subtitle text information for each of the segments, wherein the NLP language model is previously trained to differentiate between subtitle text information from video content items that were each previously identified as being an advertisement in comparison to subtitle text information from video content items that were each previously identified as not being an advertisement. The embedding representing the subtitle text information for each of the segments is passed through a classifier to obtain a probability regarding whether each segment is an advertisement or not.
H04N 21/845 - Structuring of content, e.g. decomposing content into time segments
16.
Automated identification of product or brand-related metadata candidates for a commercial using consistency between audio and image elements of products or brands detected in commercials
A method and system are provided for assigning metadata candidates to a commercial by performing image analysis on a plurality of the video frames to identify video frames that include one or more of identifiable product-related logos, brand-related logos, product-related text, or brand-related text which appear in the video frames, performing audio analysis on the commercial to identify audio associated with one or more identifiable products or brands in the commercial, comparing the products and brands identified by the image analysis with the products and brands identified by the audio analysis, and identifying similar products and brands; and assigning to the commercial a product or brand as a primary metadata candidate when a similar product or brand is identified by the image analysis and the audio analysis.
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
Systems and methods are provided to generate a personalized video channel for a viewer. Selected parameters of the viewer's viewing behavior with respect to one or more media devices that are associated with the viewer are automatically monitored and detected during one or more sampling periods. Media content sources that are available to the viewer on the one or more media devices are automatically detected. A personalized video channel is then generated for the viewer using at least the monitoring parameters and the detected media content sources. During a viewing session on one of the media devices, the generated personalized video channel is displayed.
H04N 21/466 - Learning process for intelligent management, e.g. learning user preferences for recommending movies
H04N 21/442 - Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed or the storage space available from the internal hard disk
H04N 21/24 - Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth or upstream requests
H04N 21/458 - Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules
H04N 21/234 - Processing of video elementary streams, e.g. splicing of video streams or manipulating MPEG-4 scene graphs
H04N 21/438 - Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving MPEG packets from an IP network
H04N 21/25 - Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication or learning user preferences for recommending movies
H04N 21/258 - Client or end-user data management, e.g. managing client capabilities, user preferences or demographics or processing of multiple end-users preferences to derive collaborative data
H04N 21/262 - Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission or generating play-lists
Methods and systems are provided to create a media consumer-personalized database table of selectable actions related to media content of interest to a media consumer that is being displayed on a media device. The media content of interest is identified by the media consumer by recording an indication of the media content of interest via a signal received from a user input device when the media content of interest is being displayed or presented. The database table of selectable actions is then presented to the media consumer who can initiate one of the actions.
H04N 5/445 - Receiver circuitry for displaying additional information
H04N 21/462 - Content or additional data management e.g. creating a master electronic program guide from data received from the Internet and a Head-end or controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabi
H04N 21/8545 - Content authoring for generating interactive applications
H04N 21/482 - End-user interface for program selection
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
19.
Content recommendation system and method-based implicit ratings
A content recommendation system that recommends content based on previous user content information and information regarding the user and user device. The system is capable of accurately recommending content using collaborative filtering by comparing the user's profile to other user profiles. User profiles rely on implicit ratings to determine user preferences. Implicit ratings may be determined by collecting the content that a user consumed, in addition to user and user device information. The system recommends content that similar users have consumed. The system is also able to identify and recommend trending content by analyzing changes in metrics over time. The system may also rely on automatic content recognition to identify content that the user is consuming without requiring active user input.
H04N 7/16 - Analogue secrecy systems; Analogue subscription systems
H04N 21/466 - Learning process for intelligent management, e.g. learning user preferences for recommending movies
H04N 21/45 - Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies
G06F 16/78 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
G06F 16/28 - Databases characterised by their database models, e.g. relational or object models
H04N 21/84 - Generation or processing of descriptive data, e.g. content descriptors
20.
Bidding agent using ad opportunity source to limit ad reach
A method for managing a bidding process by keeping track of the sources for ad auction opportunity on the basis of an app generating the opportunity or web activity generating the opportunity. The bidding platform may limit bids on opportunities for a campaign when the successful placement of ads or opportunities from any source exceeds the source threshold set for that source.
A method for maximizing the number of users who receive an optimal number of ad impressions. The method may be implemented for a bidding agent or in connection with a system for augmenting ad placement data across bidding platforms that bid on ad opportunities in a real-time bidding ad exchange. The system is able to use tracking pixels in order to optimize bid placements. The system is intended to manage user ad placement so as to increase the number of placements received by individual users so that the campaign has an increased impact on eligible users and reduces the diluted exposures to users who receive less than optimal exposure.
A closed loop attribution system may include a user location history storage system containing information indicative of user location history for a plurality of users based on location of a mobile device associated with a user; a user database storage system containing placement tracking information indicative of advertising content presented to the user; and a closed loop attribution processor responsive to said user location history storage system and said user database storage assessing correlations between a user's exposure and a user's location. The user database may include records indicative of user behavior and characteristics. The closed loop attribution processor may be connected to the campaign database and the campaign database may contain an indication of one or more locations of interest to an advertiser. The correlation between user exposure and a user's location may be a correlation between user exposure and the location or locations of interest.
A method is provided to classify whether video content is likely to be an advertisement or a non-advertisement. A curated database of video content items that includes a plurality of different video content items that were each previously identified as being an advertisement, and a plurality of different video content items that were each previously identified as not being an advertisement, are used to train a 2D CNN and a 3D CNN. The training of the 2D CNN includes learning characteristic visual and spatial features of advertisement images in the video content items compared to non-advertisement images in the video content items, the training resulting in weights being defined for the 2D CNN. The training of the 3D CNN includes learning a temporal structure and relationship over multiple image frames of the advertisements in the video content items compared to non-advertisement image frames in the video content items, the training resulting in weights being defined for the 3D CNN. The trained 2D CNN and 3D CNN are then used to determine the probability that newly identified video content should be classified as an advertisement.
H04N 21/439 - Processing of audio elementary streams
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
G06K 9/00 - Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
G06N 20/10 - Machine learning using kernel methods, e.g. support vector machines [SVM]
An artificial neural network architecture is provided for processing raw audio waveforms to create speaker representations that are used for text-independent speaker verification and recognition. The artificial neural network architecture includes a strided convolution layer, first and second sequentially connected residual blocks, a transformer layer, and a final fully connected (FC) layer. The strided convolution layer is configured to receive raw audio waveforms from a speaker. The first and the second residual blocks both include multiple convolutional and max pooling layers. The transformer layer is configured to aggregate frame level embeddings to an utterance level embedding. The output of the FC layer creates a speaker representation for the speaker whose raw audio waveforms were inputted into the strided convolution layer.
Systems and methods are provided for selectively replacing commercials that are in a video data stream with alternative commercials. Automatic content recognition (ACR) is performed on the video data stream to detect the identity of each of the commercials played in a commercial block during a commercial break. Commercials in the video data stream which are detected as being displayed on a video display device are stored in a first database, and commercials in the video data stream which are detected as not being displayed on the video display device are stored in a second database. A rules engine defines how commercials in the video data stream should be replaced with other commercials. Commercials in the video data stream that were previously detected as being displayed on the video display device are selectively replaced with commercials that were previously detected as not being displayed on the video display device.
H04N 21/234 - Processing of video elementary streams, e.g. splicing of video streams or manipulating MPEG-4 scene graphs
H04N 21/442 - Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed or the storage space available from the internal hard disk
H04N 21/433 - Content storage operation, e.g. storage operation in response to a pause request or caching operations
H04N 21/454 - Content filtering, e.g. blocking advertisements
26.
DETECTION OF POTENTIAL COMMERCIAL BY DETECTION AND ANALYSIS OF TRANSITIONS IN VIDEO CONTENT
A system and method are provided for detecting the presence of potential commercials in a video data stream. Each of the commercials has an expected time length that is an integer multiple of a first predetermined time length, and has an overall time length that is equal to or less than a second predetermined time length. Transitions in the audio or video of the video data stream are detected and the time of the transitions are recorded. Time differences between one or more successive transitions are detected. Any time differences that are an integer multiple of the first predetermined time length, and that have an overall time length that is equal to or less than the second predetermined time length are identified. The contents of the video data stream associated with the identified time differences are flagged as potential commercials and sent to a content processing platform for further analysis.
G06K 9/00 - Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
27.
AUTOMATED IDENTIFICATION OF PRODUCT OR BRAND-RELATED METADATA CANDIDATES FOR A COMMERCIAL USING PERSISTENCE OF PRODUCT OR BRAND-RELATED TEXT OR OBJECTS IN VIDEO FRAMES OF THE COMMERCIAL
A method and system are provided for assigning metadata candidates to a commercial by performing image analysis on a plurality of the video frames to identify video frames that include one or more of identifiable product-related logos, brand-related logos, product-related text, or brand-related text which appear in the video frames, capturing frame data for such video frames, calculating a persistence metric from the frame data for each of the identifiable product- related logos, brand-related logos, product-related text, or brand-related text, comparing the persistence metric for each of the identifiable product-related logos, brand-related logos, product-related text, or brand-related text to a predetermined threshold value, and assigning to the commercial, the identified products or brands as metadata candidates when the persistence metric exceeds the predetermined threshold value. The persistence metric defines a percentage of frames, or a number of consecutively analyzed frames, that include the one or more of identifiable product-related logos, brand-related logos, product-related text, or brand-related text.
A system and method are provided for detecting the presence of potential commercials in a video data stream. Each of the commercials has an expected time length that is an integer multiple of a first predetermined time length, and has an overall time length that is equal to or less than a second predetermined time length. Transitions in the audio or video of the video data stream are detected and the time of the transitions are recorded. Time differences between one or more successive transitions are detected. Any time differences that are an integer multiple of the first predetermined time length, and that have an overall time length that is equal to or less than the second predetermined time length are identified. The contents of the video data stream associated with the identified time differences are flagged as potential commercials and sent to a content processing platform for further analysis.
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
H04N 21/439 - Processing of audio elementary streams
H04N 21/454 - Content filtering, e.g. blocking advertisements
G06K 9/00 - Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
29.
SYSTEM AND METHOD FOR DETECTING REPEATING CONTENT, INCLUDING COMMERCIALS, IN A VIDEO DATA STREAM USING AUDIO-BASED AND VIDEO-BASED AUTOMATED CONTENT RECOGNITION
Methods and apparatus are provided for detecting potential repeating content, such as commercials, in a video data stream by receiving one or more video data streams, parsing each video data stream into a plurality of segments, creating audio fingerprints of each segment, storing the plurality of audio fingerprints in a database, and identifying any audio fingerprints in the received and parsed one or more video data streams that match audio fingerprints in the database that were previously stored from video data streams that were previously received and parsed. Video fingerprints are then created for these same pairs of segments and a similarity analysis is performed. The results of the video fingerprint analysis is used to make a determination of subsequent actions to be taken by a content processing platform that performs recognition processing of the content associated with the segments identified as being potentially repeating content.
G06F 16/783 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
30.
System and method for detecting repeating content, including commercials, in a video data stream using audio-based and video-based automated content recognition
Methods and apparatus are provided for detecting potential repeating content, such as commercials, in a video data stream by receiving one or more video data streams, parsing each video data stream into a plurality of segments, creating audio fingerprints of each segment, storing the plurality of audio fingerprints in a database, and identifying any audio fingerprints in the received and parsed one or more video data streams that match audio fingerprints in the database that were previously stored from video data streams that were previously received and parsed. Video fingerprints are then created for these same pairs of segments and a similarity analysis is performed. The results of the video fingerprint analysis is used to make a determination of subsequent actions to be taken by a content processing platform that performs recognition processing of the content associated with the segments identified as being potentially repeating content.
H04N 7/173 - Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
G06F 16/783 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
H04N 21/84 - Generation or processing of descriptive data, e.g. content descriptors
Unknown potential commercials are detected in a video data stream that contains segments of program type content, and blocks of commercial content. Each block includes a plurality of successive individual commercials. A library of known commercials is maintained in a first database. A video data stream is received in a video processing engine which includes a search engine that is in communication with the first database. The search engine identifies all known commercials in the video data stream and their respective start and end times. The video processing engine identifies all time segments that are sandwiched between the known commercials. The video processing engine filters out as a potential commercial any identified time segments that are significantly longer than the time length of a commercial. The video processing engine designates content of each of the time segments that were not filtered out as being one or more unknown potential commercials.
H04N 21/4545 - Input to filtering algorithms, e.g. filtering a region of the image
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
H04N 21/454 - Content filtering, e.g. blocking advertisements
H04N 21/234 - Processing of video elementary streams, e.g. splicing of video streams or manipulating MPEG-4 scene graphs
H04N 21/235 - Processing of additional data, e.g. scrambling of additional data or processing content descriptors
Methods and apparatus are provided for detecting potential repeating content, such as commercials, in a video data stream by receiving one or more video data streams, parsing each video data stream into a plurality of segments, creating a representation of each segment such as a segment fingerprint, storing the plurality of segment representations in a database, and identifying any segment representations in the received and parsed one or more video data streams that match segment representations in the database that were previously stored from video data streams that were previously received and parsed.
G06F 16/71 - Indexing; Data structures therefor; Storage structures
G06F 16/78 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
G06F 16/783 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
H04N 21/845 - Structuring of content, e.g. decomposing content into time segments
H04N 21/84 - Generation or processing of descriptive data, e.g. content descriptors
33.
Automated identification of product or brand-related metadata candidates for a commercial using dominance and prominence of product or brand-related text or objects in video frames of the commercial
A method and system are provided for assigning metadata candidates to a commercial by performing image analysis on a plurality of the video frames to identify video frames that include one or more of identifiable product-related logos, brand-related logos, product-related text, or brand-related text which appear in the video frames, capturing frame data for such video frames, calculating a dominance or prominence metric from the frame data for each of the identifiable product-related logos, brand-related logos, product-related text, or brand-related text, comparing the dominance or prominence metric for each of the identifiable product-related logos, brand-related logos, product-related text, or brand-related text to a predetermined threshold percentage or value, and assigning to the commercial, the identified products or brands as metadata candidates when the dominance or prominence metric exceeds the predetermined threshold percentage or value.
G06K 9/00 - Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
G06F 16/71 - Indexing; Data structures therefor; Storage structures
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
H04N 21/83 - Generation or processing of protective or descriptive data associated with content; Content structuring
H04N 21/431 - Generation of visual interfaces; Content or additional data rendering
34.
Automated identification of product or brand-related metadata candidates for a commercial using persistence of product or brand-related text or objects in video frames of the commercial
A method and system are provided for assigning metadata candidates to a commercial by performing image analysis on a plurality of the video frames to identify video frames that include one or more of identifiable product-related logos, brand-related logos, product-related text, or brand-related text which appear in the video frames, capturing frame data for such video frames, calculating a persistence metric from the frame data for each of the identifiable product-related logos, brand-related logos, product-related text, or brand-related text, comparing the persistence metric for each of the identifiable product-related logos, brand-related logos, product-related text, or brand-related text to a predetermined threshold value, and assigning to the commercial, the identified products or brands as metadata candidates when the persistence metric exceeds the predetermined threshold value.
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
G06F 17/30 - Information retrieval; Database structures therefor
H04N 21/83 - Generation or processing of protective or descriptive data associated with content; Content structuring
G06K 9/00 - Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
H04N 21/431 - Generation of visual interfaces; Content or additional data rendering
35.
Automated identification of product or brand-related metadata candidates for a commercial using consistency between audio and image elements of products of brands detected in commercials
A method and system are provided for assigning metadata candidates to a commercial by performing image analysis on a plurality of the video frames to identify video frames that include one or more of identifiable product-related logos, brand-related logos, product-related text, or brand-related text which appear in the video frames, performing audio analysis on the commercial to identify audio associated with one or more identifiable products or brands in the commercial, comparing the products and brands identified by the image analysis with the products and brands identified by the audio analysis, and identifying similar products and brands; and assigning to the commercial a product or brand as a primary metadata candidate when a similar product or brand is identified by the image analysis and the audio analysis.
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
G06K 9/62 - Methods or arrangements for recognition using electronic means
Methods and apparatus are provided for determining a lift metric regarding effectiveness of a digital ad campaign for an audio-visual work on subsequent viewership of the audio-visual work. Viewed content from a universe of monitored AV audio-visual devices is collected. Identifiers of audio-visual devices that received an ad impression for the audio-visual work are also collected. The lift metric may be determined from statistical analysis of this data.
H04N 21/2668 - Creating a channel for a dedicated end-user group, e.g. by inserting targeted commercials into a video stream based on end-user profiles
H04N 21/24 - Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth or upstream requests
37.
Automated methods for determining a user's likely exposure to media content being output by a media device based on user interactions with a mobile device that monitors media content outputted by the media device
Automated methods are provided for determining a user's likely exposure to media content being output by a media device based on user interactions with a mobile device that is monitoring the media content being outputted by the media device. When a user interacts with their mobile device, such as by using telephone functionality on the mobile device, or by interacting with an application on the mobile device, media content being played on a media device that is in proximity of the media device is sensed and detected using automatic content recognition technology. Media impressions are recorded for the detected media impression.
H04H 60/32 - Arrangements for monitoring conditions of receiving stations, e.g. malfunction or breakdown of receiving stations
H04N 21/442 - Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed or the storage space available from the internal hard disk
H04M 3/22 - Arrangements for supervision, monitoring or testing
Methods and apparatus are provided for automatically removing erroneously logged commercials from a listing of commercials that are detected in a video data stream by performing automatic content recognition on the video data stream and detecting the identity of each of the commercials played in a commercial break, temporarily logging the identity and start and end time of each detected commercial in a log of played commercials, forming clusters from commercials that overlap in time and have related content, or have significant overlap in time, forming permutations of commercial break timelines from the detected commercials, ranking the timelines based on best fit criteria and selecting the best fit timeline, permanently logging only the commercials in the best fit timeline, and removing the remaining commercials from the temporary log. The remaining logged commercials are presumed to be either erroneously identified commercials or properly identified commercials with erroneous start and end times.
Methods and apparatus are provided for detecting potential repeating content, such as commercials, in a video data stream by receiving one or more video data streams, parsing each video data stream into a plurality of segments, creating a representation of each segment such as a segment fingerprint, storing the plurality of segment representations in a database, and identifying any segment representations in the received and parsed one or more video data streams that match segment representations in the database that were previously stored from video data streams that were previously received and parsed.
H04N 21/4402 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
40.
System and method for detecting repeating content, including commercials, in a video data stream
Methods and apparatus are provided for detecting potential repeating content, such as commercials, in a video data stream by receiving one or more video data streams, parsing each video data stream into a plurality of segments, creating a representation of each segment such as a segment fingerprint, storing the plurality of segment representations in a database, and identifying any segment representations in the received and parsed one or more video data streams that match segment representations in the database that were previously stored from video data streams that were previously received and parsed.
G06F 17/30 - Information retrieval; Database structures therefor
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
Unknown potential commercials are detected in a video data stream that contains segments of program type content, and blocks of commercial content. Each block includes a plurality of successive individual commercials. A library of known commercials is maintained in a first database. A video data stream is received in a video processing engine which includes a search engine that is in communication with the first database. The search engine identifies all known commercials in the video data stream and their respective start and end times. The video processing engine identifies all time segments that are sandwiched between the known commercials. The video processing engine filters out as a potential commercial any identified time segments that are significantly longer than the time length of a commercial. The video processing engine designates content of each of the time segments that were not filtered out as being one or more unknown potential commercials.
Unknown potential commercials are detected in a video data stream that contains segments of program type content, and blocks of commercial content. Each block includes a plurality of successive individual commercials. A library of known commercials is maintained in a first database. A video data stream is received in a video processing engine which includes a search engine that is in communication with the first database. The search engine identifies all known commercials in the video data stream and their respective start and end times. The video processing engine identifies all time segments that are sandwiched between the known commercials. The video processing engine filters out as a potential commercial any identified time segments that are significantly longer than the time length of a commercial. The video processing engine designates content of each of the time segments that were not filtered out as being one or more unknown potential commercials.
H04N 21/845 - Structuring of content, e.g. decomposing content into time segments
H04N 21/44 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to MPEG-4 scene graphs
H04N 21/454 - Content filtering, e.g. blocking advertisements
43.
Efficient apparatus and method for audio signature generation using recognition history
Audio information is monitored by a user device that performs audio content recognition of any received audio content. The user device includes a scheduling logic unit, a probe, and an audio signature generator. The scheduling logic unit maintains a set of scheduling rules that define conditions that were present when previous audio content recognition of audio content received by the user device was successful. The scheduling logic unit receives currently present conditions of the user device, and compares the currently present conditions to the set of scheduling rules to determine if the currently present conditions match any scheduling rules. The user devices captures ambient audio content via the probe and generates audio signatures of the captured audio content using the audio signature generator if a match occurs, and inhibits capturing audio content by the user device and/or inhibits generating audio signatures if a match does not occur.
H04W 4/02 - Services making use of location information
G10L 15/06 - Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
G10L 25/03 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
G10L 25/27 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique
G10L 25/48 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use
G10L 25/51 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination
H04H 60/58 - Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups or of audio
H04W 4/021 - Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
G06F 16/683 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Methods and apparatus are provided for determining a lift metric regarding effectiveness of a digital ad campaign for an audio-visual work on subsequent viewership of the audio-visual work. Viewed content from a universe of monitored AV audio-visual devices is collected. Identifiers of audio-visual devices that received an ad impression for the audio-visual work are also collected. The lift metric may be determined from statistical analysis of this data.
H04N 21/2668 - Creating a channel for a dedicated end-user group, e.g. by inserting targeted commercials into a video stream based on end-user profiles
H04N 21/24 - Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth or upstream requests
45.
Apparatus and method for determining co-location of services using a device that generates an audio signal
Methods and apparatus are provided for generating an audio signature. An audio signal generator generates an audio signal. A speaker responsive to the audio signal generator outputs the generated audio signal. A probe including a microphone samples the generated audio signal detected by the microphone. An audio signature generator is connected to the probe and generates an audio signature from the sampled audio signal detected by the microphone. The audio signature generator generates the audio signature without requiring detection of any additional audio signal. A transmitter connected to the audio signature generator transmits the audio signature to a communications channel.
Methods and apparatus are provided for automatically removing erroneously logged commercials from a listing of commercials that are detected in a video data stream by performing automatic content recognition on the video data stream and detecting the identity of each of the commercials played in a commercial break, temporarily logging the identity and start and end time of each detected commercial in a log of played commercials, forming clusters from commercials that overlap in time and have related content, or have significant overlap in time, forming permutations of commercial break timelines from the detected commercials, ranking the timelines based on best fit criteria and selecting the best fit timeline, permanently logging only the commercials in the best fit timeline, and removing the remaining commercials from the temporary log. The remaining logged commercials are presumed to be either erroneously identified commercials or properly identified commercials with erroneous start and end times.
H04N 21/442 - Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed or the storage space available from the internal hard disk
G06F 17/30 - Information retrieval; Database structures therefor
47.
Apparatus and method for determining co-location of services using a device that generates an audio signal
An audio signature is generated by a user device by generating an audio signal indicative of the user device, outputting the audio signal indicative of the user device through a speaker of the user device, using a microphone of the user device to sense audio, generating an audio signature of the audio sensed by the microphone in the user device, and transmitting the audio signature from the user device to a communication channel. The audio signal may be used by another user device which generates its own audio signature, and which can be compared to the audio signature of the user device that originally generated the audio signal to determine if the user devices are co-located.
Audio information is monitored by a user device that performs audio content recognition of any received audio content. The user device includes a scheduling logic unit, a probe, and an audio signature generator. The scheduling logic unit maintains a set of scheduling rules that define conditions that were present when previous audio content recognition of audio content received by the user device was successful. The scheduling logic unit receives currently present conditions of the user device, and compares the currently present conditions to the set of scheduling rules to determine if the currently present conditions match any scheduling rules. The user devices captures ambient audio content via the probe and generates audio signatures of the captured audio content using the audio signature generator if a match occurs, and inhibits capturing audio content by the user device and/or inhibits generating audio signatures if a match does not occur.
G10L 25/51 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination
H04H 60/58 - Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups or of audio
G10L 15/06 - Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
G10L 19/018 - Audio watermarking, i.e. embedding inaudible data in the audio signal
G10L 25/27 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique
G10L 25/03 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters
G10L 25/48 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
49.
Apparatus and method for determining co-location of services
Co-location of devices is determined by receiving at a query engine from a device, a remote audio signature and associated remote device identification and device location data and comparing in the query engine remote audio signatures stored in a database to the remote audio signature. The remote database further stores remote device identification and device location data associated with the stored audio signatures. The query engine only performs the comparisons for remote audio signatures that have the same location data. The query engine thereby limits its query set of remote audio signatures in the database. A processor reports a remote device identification associated with the remote audio signature of the device received over the communication channel and the remote device identification associated with the stored remote audio signature when an audio signature received over the communication channel matches an audio signature stored in the database within a threshold confidence level.
G10L 25/27 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique
G10L 25/03 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters
H04H 60/58 - Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups or of audio
G10L 25/51 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
50.
Efficient apparatus and method for audio signature generation using motion
An automatic content recognition system that includes a user device for the purpose of capturing audio and generating an audio signature. The user device may be a smartphone or tablet. The system is also capable of determining whether a user device is in motion and refraining from audio monitoring and/or generating audio signatures when the user device is in motion. Motion may also be used to reduce the frequency of audio monitoring and/or signature generation. The system may have a database within the user device or the user device may communicate with a server having a database that contains reference audio signatures.
G10L 25/27 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique
G10L 25/03 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters
G10L 25/51 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination
G10L 25/48 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
51.
Efficient apparatus and method for audio signature generation using recognition history
Audio information is monitored by a user device by generating audio signatures of ambient audio content captured by the user device, receiving conditions of the user device that were present when the ambient audio content for each audio signature was captured, performing audio content recognition on the audio signatures by comparing in a query engine the generated audio signatures to audio signatures previously stored in a database associated with the query engine, identifying generated audio signatures that match audio signatures stored in the database within a threshold confidence level, thereby establishing successful audio content recognition, identifying the conditions of the user device that were present for successful audio content recognition, generating scheduling rules regarding the conditions present during successful audio content recognition and controlling when the user device captures audio content and/or generates audio signatures based on the scheduling rules.
H04H 60/58 - Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups or of audio
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
G10L 25/51 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination
52.
Apparatus and method for determining audio and/or visual time shift
An system for determining time offset of an audio signature from a reference signature time stamp may have a server connected to one or more remote devices. The server may have a receiver connected to one of more communication channels configured to receive an audio signature generated by the remote device and transmitted over the communication channel. The system may have a database at or connected to the server that contains one or more reference audio signatures. Each of the reference audio signatures may have a time stamp also stored in the database. A query engine may compare the remote audio signature to one or more reference audio signatures stored in the database. A processor may be provided to compare a reference timestamp to a timestamp associated with said remote audio signature. The system may be used to evaluate viewing habits and particularly delayed viewing and program playback manipulation such as Fast-forward, slow-motion, skip, rewind and program abandonment.
G10L 25/27 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique
G10L 25/03 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters
G06F 16/683 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
G06F 16/955 - Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
G10L 25/51 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
H04H 60/58 - Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups or of audio
G10L 25/48 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use
G10L 19/018 - Audio watermarking, i.e. embedding inaudible data in the audio signal
G10L 15/06 - Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
An automatic content recognition system that includes a user device for the purpose of capturing audio and generating an audio signature. The user device may be a smartphone or tablet. The system is also capable of determining sound level at a user device and refraining from audio monitoring and/or generating audio signatures when the sound level is below a threshold. Sound level may also be used to reduce the frequency of audio monitoring and/or signature generation. The system may have a database within the user device or the user device may communicate with a server having a database that contains reference audio signatures.
G10L 25/27 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique
G10L 25/03 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters
G10L 25/51 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use for comparison or discrimination
G10L 25/48 - Speech or voice analysis techniques not restricted to a single one of groups specially adapted for particular use
G10L 25/18 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
54.
Method and system for targeted content distribution using tagged data streams
A method for targeting content to a user based on a user profile and content tags. All content is delivered to the user's computer system. Each unit of content has a tag uniquely identifying a classification of the content. The receiving computer recognizes and evaluates the tags to match the content with a local user profile. The matching content is selected from all the content and displayed. The user profile is a set of data describing the classifications of content that are most appropriate for the user. The profile can be developed based on usage history or demographic information. The user computer system has a tag dictionary that may be updated to correspond to a set of tags utilized by content delivery systems.
G06F 15/16 - Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
Methods and systems provide clickstream captures at both local and remote components by receiving user commands generated by the viewer locally where at least some of those commands are then sent to the remote component concurrently relative to when they are generated by the viewer. Commands may be captured and stored locally while the commands that are sent to the remote component are captured and then stored remotely from the premises of the viewer. Additionally, the user commands may be executed locally such as at a viewer appliance and/or remotely such as at a video control system of a television network. Information related to these user commands continues to be stored locally and/or remotely from the premises of the viewer after the user commands have been executed. Thus, the information related to these user commands can be processed as necessary and used for various purposes.
Methods and systems provide clickstream captures at a remote component, such as a video control system of a television network, by receiving user commands generated by the viewer that are sent to the remote component concurrently relative to when they are generated by the viewer. The commands are captured and then information related to these commands is stored remotely from the premises of the viewer. Additionally, the user commands may be executed remotely such as at a video control system of a television network or alternatively may be executed at the premises of the viewer such as within a set top box. Information related to these user commands continues to be stored remotely from the premises of the viewer after the user commands have been executed such that the information related to these user commands can be processed as necessary and used for various purposes.
H04H 60/33 - Arrangements for monitoring the users' behaviour or opinions
H04H 60/45 - Arrangements for identifying or recognising characteristics with a direct linkage to broadcast information or to broadcast space-time, e.g. for identifying broadcast stations or for identifying users for identifying users
H04N 5/445 - Receiver circuitry for displaying additional information
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
G06F 13/00 - Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
57.
Method and apparatus for providing incentives for viewers to watch commercial advertisements
A user is prompted during television commercial breaks to depress one or more pushbuttons on a television remote control at certain times to confirm that the user is watching the advertisements. A set top box connected to the television receives and decodes the signal from the remote control. When the user responds properly to a prompt, the set top box increments a tally maintained in the set top box memory to accumulate points by which the user can earn rewards.
patents
