DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 08/26/2024 have been considered by the examiner and been placed of record in the file.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
Claims 1-20 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1-18 of U.S. Patent No. 12,141,842 B2. Although the conflicting claims are not identical, they are not patentably distinct from each other. The following table shows the similarities between the two claimed inventions.
Instant application
Patent No. 12,141,842 B2
Claim 1.
A method for brand recognition in a video stream, comprising:
receiving, by a computing device, the video stream comprising a series of digital images;
automatically extracting, by the computing device, an object of interest associated with one or more brand features from the series of digital images by applying a trained brand recognition model that comprises a first neural network to one or more of the digital images to detect the one or more brand features, wherein the object of interest is associated with a brand;
creating, by the computing device, a report of results comprising the brand associated with the object of interest extracted by the trained brand recognition model and at least one time information related to the detection of the object of interest in the video stream; and
providing the report of results as an output from the computing device.
Claim 1.
A method for brand recognition in a video stream to be performed by a computing device, the method comprising:
receiving, by a brand recognition application executing on the computing device, the video stream comprising a series of digital images;
automatically extracting, by the brand recognition application, an object of interest associated with one or more brand features from the series of digital images by applying a trained brand recognition model that comprises a first neural network to each of the digital images to detect the one or more brand features, wherein the object of interest is associated with a brand and wherein automatically extracting the object of interest further comprises:
determining a tensor map based on an output of the first neural network; and determining, based on the tensor map, a region proposal comprising one or more regions, wherein the one or more regions are bounded;
creating, by the computing device, a report of results comprising the brand associated with the object of interest extracted by the trained brand recognition model and at least one time information related to the detection of the object of interest in the video stream; and
providing the report of results as an output from the computing device.
The similarities between the two independent claims are shown in bold face. The differences are minimal as a result all limitation instant application can be obtained from the parent case, Patent No. US 12,141,842 B2.
Dependent claims:
Claim 2: all limitations can be obtained from claim 2 of Patent No. US 12141842 B2.
Claim 3: all limitations can be obtained from claim 1 of Patent No. US 12141842 B2.
Claims 4-9: all limitations can be obtained respectively from claim 3-8 of Patent No. US 12141842 B2.
Claim 10.
A data processing system comprising a processor and a non-transitory data storage comprising computer-readable instructions that, when executed by the processor, perform an automated process comprising:
receiving a video stream comprising a series of digital images;
automatically extracting an object of interest associated with one or more brand features from the series of digital images by applying a trained brand recognition model that comprises a first neural network to one or more of the digital images to detect the one or more brand features, wherein the object of interest is associated with a brand;
creating a report of results comprising the brand associated with the object of interest extracted by the trained brand recognition model and at least one time information related to the detection of the object of interest in the video stream; and providing the report of results as an output from the data processing system.
Claim 9.
A data processing system comprising a processor and a non-transitory data storage comprising computer-readable instructions that, when executed by the processor, perform an automated process comprising:
receiving, by a brand recognition application, a video stream comprising a series of digital images;
automatically extracting, by the brand recognition application, an object of interest associated with one or more brand features from the series of digital images by applying a trained brand recognition model that comprises a first neural network to each of the digital images to detect the one or more brand features, wherein the object of interest is associated with a brand and wherein automatically extracting the object of interest further comprises:
determining a tensor map based on an output of the first neural network; and determining, based on the tensor map, a region proposal comprising one or more regions, wherein the one or more regions are bounded;
creating a report of results comprising the brand associated with the object of interest extracted by the trained brand recognition model and at least one time information related to the detection of the object of interest in the video stream; and providing the report of results as an output from the data processing system.
The similarities between the two independent claims are shown in bold face. The differences are minimal as a result all limitation instant application can be obtained from the parent case, Patent No. US 12,141,842 B2.
Dependent claims:
Claim 11: all limitations can be obtained from claim 10 of Patent No. US 12141842 B2.
Claim 12: all limitations can be obtained from claim 9 of Patent No. US 12141842 B2.
Claims 13-18: all limitations can be obtained respectively from claims 11-16 of Patent No. US 12141842 B2.
Claim 19.
An automated process to be performed by a data processing system to automatically recognize brand imagery in a video stream, the automated process comprising:
receiving the video stream comprising a series of digital images;
automatically extracting an object of interest associated with one or more brand features from the series of digital images by applying a trained brand recognition model that comprises a first neural network to one or more of the digital images to detect one or more brand features, wherein the object of interest is associated with a brand;
creating a report of results comprising the brand associated with the object of interest extracted by the trained brand recognition model and at least one time information related to the detection of the object of interest in the video stream; and
providing the report of results as an output from the data processing system.
Claim 17.
An automated process to be performed by a data processing system to automatically recognize brand imagery in a video stream, the automated process comprising:
receiving, by a brand recognition application executing on the data processing system, video stream comprising a series of digital images; automatically extracting, by the brand recognition application, an object of interest associated with one or more brand features from the series of digital images by applying a trained brand recognition model that comprises a first neural network to each of the digital images to detect one or more brand features, wherein the object of interest is associated with a brand and wherein automatically extracting the object of interest further comprises:
determining a tensor map based on an output of the first neural network; and
determining, based on the tensor map, a region proposal comprising one or more regions, wherein the one or more regions are bounded;
creating a report of results comprising the brand associated with the object of interest extracted by the trained brand recognition model and at least one time information related to the detection of the object of interest in the video stream; and
providing the report of results as an output from the data processing system.
The similarities between the two independent claims are shown in bold face. The differences are minimal as a result all limitation instant application can be obtained from the parent case, Patent No. US 12,141,842 B2.
Dependent claims:
Claim 20: all limitations can be obtained from claim 18 of Patent No. US 12141842 B2.
It would have been obvious to a person of ordinary skill in the art, at the time the invention was made to use claims of US Patent No. US 12,141,842 B2 to provide all the functions of the current application 18/815,188.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, 5-9, 10-11, 14-10 and 20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
As to Independent claims 1, 10 and 19:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
For claim 1, Yes, the claim is a process.
For claim 10, Yes, the claim is a machine.
For claim 19, Yes, the claim is a process.
Step 2A Prong One Analysis: Do the claims recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the claims recite an abstract idea (mental process) of analyzing image data using an algorithm. The claims disclose the idea of brand recognition including the steps of receiving video steam, extracting object of interest using neural networks and creating and proving a report. All the steps of the method recited in the claims are readily capable of being performed in the human mind or using pen and paper. Automating a mental process does not overcome 101 rejection as abstract idea. Therefore these steps, in claims 1, 10 and 19, are each a mental process.. See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitations “applying a trained brand recognition model that comprises a first neural network to one or more of the digital images to detect the one or more brand features” or “computer-readable medium storing instructions that when executed by at least one processor cause the at least one processor to execute operations, the operations comprising” recited in independent claims 1 and 10 respectively are additional elements that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer or neural network model in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
Analysis applied to claim 1 is applicable to claim 19.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitations “applying a trained brand recognition model that comprises a first neural network to one or more of the digital images to detect the one or more brand features” or “computer-readable medium storing instructions that when executed by at least one processor cause the at least one processor to execute operations, the operations comprising” recited in independent claims 1 and 10 respectively are additional elements that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer or neural network model in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
Analysis applied to claim 1 is applicable to claim 19.
As to claims 2, 5, 11, 14 and 20:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
For claims 2, 5 Yes, the claims are processes.
For claims 11, 14 and 20, Yes, the claims are machines.
Step 2A Prong One Analysis: Do the claims recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the claims included limitations identified as an abstract idea in the parent claims.
Step 2A Prong Two Analysis: Do the claims recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, all elements are part of the abstract idea as shown above. Defining the neural being used, CNN or R-CNN, are tools being used to perform existing mental process.
Step 2B Analysis: Do the claims recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, all elements are part of the abstract idea as shown above. Defining the neural being used, CNN or R-CNN, are tools being used to perform existing mental process.
As to claims 6 and 15:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
For claim 6, Yes, the claim is a process.
For claim 15 Yes, the claim is a machine
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the claims included limitations identified as an abstract idea in the parent claims.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, all elements are part of the abstract idea as shown above.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, all elements are part of the abstract idea as shown above.
As to claims 7-9 and 16-18:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
For claims 7-9, Yes, the claims are processes.
For claims 16-18 Yes, the claims are machines.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the claims included limitations identified as an abstract idea in the parent claims.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, all elements are part of the abstract idea as shown above
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, all elements are part of the abstract idea as shown above. . Identifying the type of data being included in the report or the video stream is from broadcast do not add anything more than the abstract idea, no does it make them as being integrated in a practical application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 6-10 and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Song et al. (US 10970577 B1) in view of Venkataswamy et al. (US 9148708 B2).
Claim 1. Song et al. disclose A method for brand recognition in a video stream (FIG. 1-11), comprising:
receiving, by a computing device (read as server computer or client device (Column 5 lines 45-46)), the video stream comprising a series of digital images (read as receive messages that include text and media content such as pictures and video (Column 6 lines 17-19) … directly receives the one or more images or the video stream captured by the image capture device (Column 7 lines 33-34));
automatically extracting, by the computing device, an object of interest associated with one or more brand features from the series of digital images by applying a trained brand recognition model that comprises a first neural network to one or more of the digital images to detect the one or more brand features, wherein the object of interest is associated with a brand (read as The deep learning model may be provided training materials and use machine learning techniques to identify features which indicate an object of interest, such as a company logo (Column 7 lines 63-66));
creating, by the computing device, a report of results comprising the brand associated with the object of interest extracted by the trained brand recognition model and at least one time information related to the detection of the object of interest in the video stream (read as The extracted features may be incorporate the low-level features identified by the region component 220, text, shapes, and any other suitable information. The extracted features may be associated with semantic representations (e.g., semantic features) to enable comparative analysis of the extracted features (Column 8 lines 12-17)); and
providing the report of results as an output from the computing device.
Song et al. do not explicitly disclose creating and providing a report as an output.
However, in the related field of endeavor Venkataswamy et al. disclose: … The report generation module 220 may be configured to generate a statistical report depicting the number of times the at least one electronic message is appended in the one or more video frames 302-1 . . . 302-N (Column 9 lines ). The idea, of generating and providing a report on items included in a video stream, is clearly disclosed by Venkataswamy et al.
Therefore, it would have been obvious to a person of ordinary skill in the art, at the time the invention was filed, to modify the teaching of Song et al. with the teaching of Venkataswamy et al. in order to provide an effective and efficient mechanism for detecting the presence of the at least one content using signal processing techniques such as image/video/multimedia processing techniques (Venkataswamy et al. (Column 4 lines 1-4)).
Claim 6. The method of claim 1, the combination of Song et al. and Venkataswamy et al. teaches,
further comprising training the trained brand recognition model by:
receiving a training set of one or more images in a training video stream comprising brand images with one or more brand features associated with a brand object (Song et al. : read as A training set for the deep learning model may include a number of logo or icon classes for logos or icons registered with the database, a number of annotated images, and diverse and complex backgrounds (Column 11 lines 58-61); and
tagging the one or more images of the training set (Venkataswamy et al.: read as … one or more annotated frames 414. The one or more annotated frames 414 comprising the one or more video frames 404-1, 404-2 . . . 404-N (Column 9 lines 44-46)).
Claim 7. The method of claim 1, the combination of Song et al. and Venkataswamy et al. teaches,
wherein:
the report of results is a table; and
the time information comprises at least one selected from the group of a number of appearances of the object of interest, a number of appearances of the brand, a total display time of the object of interest, a total display time of the brand, a frequency of display of the object of interest, and a frequency of display of the brand (Venkataswamy et al.: read as The report generation module 220 may be configured to generate a statistical report depicting the number of times the at least one electronic message is appended in the one or more video frames 302-1 . . . 302-N (Column 9 lines ). The specific items being claimed are not explicitly disclosed Venkataswamy et al. However, it is obvious that such quantities can be included in the statistical report disclosed by Venkataswamy et al. ).
Claim 8. The method of claim 7, the combination of Song et al. and Venkataswamy et al. teaches,
wherein the report of results further comprises a category of the brand (read as The classification layer may enable a classification process to work as a matching process, inferring an image belongs to a closest class based on the semantic features… a region of a shoe containing a NIKE® icon, when projected into the n-dimensional metric space, will be closer to a NIKE® logo than a STARBUCKS® logo (Column 9 lines 45-51 … 60-65)).
Claim 9. The method of claim 1, the combination of Song et al. and Venkataswamy et al. teaches,
wherein the video stream comprises a video broadcast (Venkataswamy et al.: read as at least one video stream broadcasted by at least one broadcasting channel (Column 2 lines 49-50)).
Claim 10. Song et al. disclose A data processing system (FIG. 1) comprising a processor and a non-transitory data storage comprising computer-readable instructions that, when executed by the processor (read as a server computer system) or hardware components of a computer system (e.g., at least one hardware processor, a processor, or a group of processors) is configured by software (Column 12 lines 39-42)), perform an automated process comprising:
receiving a video stream comprising a series of digital images (read as receive messages that include text and media content such as pictures and video (Column 6 lines 17-19) … directly receives the one or more images or the video stream captured by the image capture device (Column 7 lines 33-34));
automatically extracting an object of interest associated with one or more brand features from the series of digital images by applying a trained brand recognition model that comprises a first neural network to one or more of the digital images to detect the one or more brand features, wherein the object of interest is associated with a brand (read as The deep learning model may be provided training materials and use machine learning techniques to identify features which indicate an object of interest, such as a company logo (Column 7 lines 63-66));
creating a report of results comprising the brand associated with the object of interest extracted by the trained brand recognition model and at least one time information related to the detection of the object of interest in the video stream (read as The extracted features may be incorporate the low-level features identified by the region component 220, text, shapes, and any other suitable information. The extracted features may be associated with semantic representations (e.g., semantic features) to enable comparative analysis of the extracted features (Column 8 lines 12-17)); and
providing the report of results as an output from the data processing system.
providing the report of results as an output from the computing device.
Song et al. do not explicitly disclose creating and providing a report as an output.
However, in the related field of endeavor Venkataswamy et al. disclose: … The report generation module 220 may be configured to generate a statistical report depicting the number of times the at least one electronic message is appended in the one or more video frames 302-1 . . . 302-N (Column 9 lines ). The idea, of generating and providing a report on items included in a video stream, is clearly disclosed by Venkataswamy et al.
Therefore, it would have been obvious to a person of ordinary skill in the art, at the time the invention was filed, to modify the teaching of Song et al. with the teaching of Venkataswamy et al. in order to provide an effective and efficient mechanism for detecting the presence of the at least one content using signal processing techniques such as image/video/multimedia processing techniques (Venkataswamy et al. (Column 4 lines 1-4)).
Claim 15. The data processing system of claim 10, the combination of Song et al. and Venkataswamy et al. teaches,
wherein the automated process further comprises:
training the trained brand recognition model by:
receiving a training set of one or more images in a training video stream comprising brand images with one or more brand features associated with a brand object (Song et al. : read as A training set for the deep learning model may include a number of logo or icon classes for logos or icons registered with the database, a number of annotated images, and diverse and complex backgrounds (Column 11 lines 58-61); and
tagging the one or more images of the training set (Venkataswamy et al.: read as … one or more annotated frames 414. The one or more annotated frames 414 comprising the one or more video frames 404-1, 404-2 . . . 404-N (Column 9 lines 44-46)).
Claim 16. The data processing system of claim 10, the combination of Song et al. and Venkataswamy et al. teaches,
wherein:
the report of results is a table (Venkataswamy et al.: read as The channel logo database 224 may comprise a table with multiple rows and columns, wherein the columns include channel name, logo, features of the logo, and category of the channel, etc. that can be maintained. (Column 7 lines 27-30)); and
the time information comprises at least one selected from the group of a number of appearances of the object of interest, a number of appearances of the brand, a total display time of the object of interest, a total display time of the brand, a frequency of display of the object of interest, and a frequency of display of the brand (Venkataswamy et al.: read as The report generation module 220 may be configured to generate a statistical report depicting the number of times the at least one electronic message is appended in the one or more video frames 302-1 . . . 302-N (Column 9 lines ). The specific items being claimed are not explicitly disclosed Venkataswamy et al. However, it is obvious that such quantities can be included in the statistical report disclosed by Venkataswamy et al.).
Claim 17. The data processing system of claim 16, the combination of Song et al. and Venkataswamy et al. teaches,
wherein the report of results further comprises a category of the brand (read as The classification layer may enable a classification process to work as a matching process, inferring an image belongs to a closest class based on the semantic features… a region of a shoe containing a NIKE® icon, when projected into the n-dimensional metric space, will be closer to a NIKE® logo than a STARBUCKS® logo (Column 9 lines 45-51 … 60-65)).
Claim 18. The data processing system of claim 10, the combination of Song et al. and Venkataswamy et al. teaches,
wherein the video stream comprises a video broadcast (Venkataswamy et al.: read as at least one video stream broadcasted by at least one broadcasting channel (Column 2 lines 49-50)).
Claim 19. Song et al. disclose An automated process to be performed by a data processing system to automatically recognize brand imagery in a video stream (FIG. 1-11), the automated process comprising:
receiving the video stream comprising a series of digital images (read as receive messages that include text and media content such as pictures and video (Column 6 lines 17-19) … directly receives the one or more images or the video stream captured by the image capture device (Column 7 lines 33-34));
automatically extracting an object of interest associated with one or more brand features from the series of digital images by applying a trained brand recognition model that comprises a first neural network to one or more of the digital images to detect one or more brand features, wherein the object of interest is associated with a brand (read as The deep learning model may be provided training materials and use machine learning techniques to identify features which indicate an object of interest, such as a company logo (Column 7 lines 63-66));
creating a report of results comprising the brand associated with the object of interest extracted by the trained brand recognition model and at least one time information related to the detection of the object of interest in the video stream (read as The extracted features may be incorporate the low-level features identified by the region component 220, text, shapes, and any other suitable information. The extracted features may be associated with semantic representations (e.g., semantic features) to enable comparative analysis of the extracted features (Column 8 lines 12-17)); and
providing the report of results as an output from the data processing system.
Song et al. do not explicitly disclose creating and providing a report as an output.
However, in the related field of endeavor Venkataswamy et al. disclose: … The report generation module 220 may be configured to generate a statistical report depicting the number of times the at least one electronic message is appended in the one or more video frames 302-1 . . . 302-N (Column 9 lines ). The idea, of generating and providing a report on items included in a video stream, is clearly disclosed by Venkataswamy et al.
Therefore, it would have been obvious to a person of ordinary skill in the art, at the time the invention was filed, to modify the teaching of Song et al. with the teaching of Venkataswamy et al. in order to provide an effective and efficient mechanism for detecting the presence of the at least one content using signal processing techniques such as image/video/multimedia processing techniques (Venkataswamy et al. (Column 4 lines 1-4)).
Claims 2-3, 11-12 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Song et al. Song et al. (US 10970577 B1) and Venkataswamy et al. (US 9148708 B2) in view of Polak et al. (US 2019/0294881 A1).
Claim 2. The method of claim 1, the combination of Song et al. and Venkataswamy et al. does not explicitly disclose,
wherein the first neural network is a trained neural network comprising a convolutional neural network (CNN).
However, in the related field of endeavor Polak et al. disclose: the first set of trained classification functions comprises one or more convolutional neural networks (CNN) organized with hierarchical classes [0023].
Therefore, it would have been obvious to a person of ordinary skill in the art, at the time the invention was filed, to modify the teaching of the combination of Song et al. and Venkataswamy et al. with the teaching of Polak et al. in order to recognize behavior, and, more specifically, but not exclusively, to recognize behavior of one or more objects detected in a video stream (Polak et al. [0002]).
Claim 3. The method of claim 1, the combination of Song et al. and Venkataswamy et al. does not explicitly disclose,
wherein automatically extracting an object of interest further comprises:
determining a tensor map based on the output of the first neural network; and
determining, based on the tensor map, a region proposal comprising one or more regions, wherein the one or more regions are bounded.
However, in the related field of endeavor Polak et al. disclose: … the analyzed frame is divided to equally fixed size grid cells, for example, N×N grid cells and mapped to a respective tensor having dimensions of N×N×M, i.e. the tensor comprises N×N attribute sets where M is the total number of object classes and their respective attributes … [0086].
Therefore, it would have been obvious to a person of ordinary skill in the art, at the time the invention was filed, to modify the teaching of the combination of Song et al. and Venkataswamy et al. with the teaching of Polak et al. in order to recognize behavior, and, more specifically, but not exclusively, to recognize behavior of one or more objects detected in a video stream (Polak et al. [0002]).
Claim 11. The data processing system of claim 10, the combination of Song et al. and Venkataswamy et al. does not explicitly disclose,
wherein the first neural network is a trained neural network comprising a convolutional neural network (CNN).
However, in the related field of endeavor Polak et al. disclose: the first set of trained classification functions comprises one or more convolutional neural networks (CNN) organized with hierarchical classes [0023].
Therefore, it would have been obvious to a person of ordinary skill in the art, at the time the invention was filed, to modify the teaching of the combination of Song et al. and Venkataswamy et al. with the teaching of Polak et al. in order to recognize behavior, and, more specifically, but not exclusively, to recognize behavior of one or more objects detected in a video stream (Polak et al. [0002]).
Claim 12. The data processing system of claim 10, the combination of Song et al. and Venkataswamy et al. does not explicitly disclose,
wherein automatically extracting the object of interest further comprises:
determining a tensor map based on the output of the first neural network; and
determining, based on the tensor map, a region proposal comprising one or more regions, wherein the one or more regions are bounded.
However, in the related field of endeavor Polak et al. disclose: … the analyzed frame is divided to equally fixed size grid cells, for example, N×N grid cells and mapped to a respective tensor having dimensions of N×N×M, i.e. the tensor comprises N×N attribute sets where M is the total number of object classes and their respective attributes … [0086].
Therefore, it would have been obvious to a person of ordinary skill in the art, at the time the invention was filed, to modify the teaching of the combination of Song et al. and Venkataswamy et al. with the teaching of Polak et al. in order to recognize behavior, and, more specifically, but not exclusively, to recognize behavior of one or more objects detected in a video stream (Polak et al. [0002]).
Claim 20. The automated process of claim 19, the combination of Song et al. and Venkataswamy et al. does not explicitly disclose,
wherein the first neural network is a convolutional neural network (CNN) configured to classify the object of interest based upon the associated one or more brand features, and wherein the automated process further comprises detecting a location of the object of interest within the digital images of the video stream using a region convolutional neural network (R-CNN) separate from the CNN.
However, in the related field of endeavor Polak et al. disclose: the first set of trained classification functions comprises one or more convolutional neural networks (CNN) organized with hierarchical classes [0023].
Therefore, it would have been obvious to a person of ordinary skill in the art, at the time the invention was filed, to modify the teaching of the combination of Song et al. and Venkataswamy et al. with the teaching of Polak et al. in order to recognize behavior, and, more specifically, but not exclusively, to recognize behavior of one or more objects detected in a video stream (Polak et al. [0002]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892.
Additional prior art included in PTO-892 disclose ideas related to the claimed invention. In this regard, Chaudhari et al. (US 2021/0012145 A1) disclose: System 10 may process a single image or a series of images to determine from among such input a presence of an ad, category, or brand, e.g., by using a last or penultimate frame in a series of frames of a GIF. The GIF may then be converted into static images, and these images may then be run through method 300 of FIG. 5 to recognize shapes, categories of shapes, brands, text, etc.[0031]. FIG. 5.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOHAMMED RACHEDINE whose telephone number is (571)272-9249. The examiner can normally be reached Mon-Fri 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jeanette J. Parker can be reached at (571)270-3647. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MOHAMMED . RACHEDINE
Examiner
Art Unit 2649
/MOHAMMED RACHEDINE/Primary Examiner, Art Unit 2646
Prosecution Insights