DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on June 29, 2026 has been entered.
Response to Amendment
Receipt is acknowledged of claim amendments with associated arguments/remarks, received June 29, 2026. Claims 1-20 are pending with amendments to claims 1, 5, 10, 14, 17.
Response to Arguments
Applicant’s arguments, see pg 8, filed June 29, 2026, with respect to the rejections of claim 1-20, under 35 USC § 103 has been fully considered and is persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new grounds of rejection is made under Boskovich (US 20190238952) in view of Shihadah et al (US 2013/0282532).
All arguments were addressed.
Claim Rejections - 35 USC § 112(a) – New Matter
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The following claims each contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 10, 17 were each amended to claim “wherein the consumption data indicates whether a threshold number of other users of the platform have performed an action on a portion of at least one of the set of media items that depicts the one or more objects over a time period”
The applicant cites support for the amendment from the specification ¶ [0038]-[0041] in the Remarks – 06/29/2026, pg 8. However, the specification does not provide support for the as claimed by the applicant. The examiner reviewed the entire specification and did not find support. Regarding a threshold, the specification states “Some or all of the data (e.g., measurements) may be weighted and used to generate a score (e.g., object preference score) and the score of an object may be compared to a threshold value. One or more of the objects with a score that satisfies the threshold (above or below threshold) may be chosen for annotation and source linking.” at paragraph [0041]. The specification does not support the language as amended by the applicant. The claim amendment was not presented in the original submitted claims. Therefore, the claim limitation represents a new matter not supported by the specification.
Claims 2-9 are rejected as dependent on claim 1.
Claims 11-16 are rejected as dependent on claim 10.
Claims 18-20 are rejected as dependent on claim 17.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 10, 17 were each amended to claim “wherein the consumption data indicates whether a threshold number of other users of the platform have performed an action on a portion of at least one of the set of media items that depicts the one or more objects over a time period”
The limitation is indefinite as it is unclear if the threshold number of other users is analyzed with respect to a performing an action (in general) and the media depicts the objects over a time period or if the threshold number of other users is analyzed with respect to a performing the action within a time period. As discussed above, the specification does not readily provide guidance on interpretation of the claim limitation. The Remarks do not provide details for interpretation (Remarks – 06/29/2026, pg 7). The After-Final Examiner Interview Agenda did not provide details (06/16/2026) and the examiner noted additional time to review the amendment was necessary. Thus, Applicant has failed to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claims 2-9 are rejected as dependent on claim 1.
Claims 11-16 are rejected as dependent on claim 10.
Claims 18-20 are rejected as dependent on claim 17.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Boskovich (US 20190238952) in view of Shihadah et al (US 2013/0282532, disclosed in IDS 01/26/2026).
Regarding Claim 1, Boskovich teach a method (implementation 1200 of system 101; Fig 1, 11-12 and ¶ [0323]-[0329]) comprising:
identifying a set of media items accessible to users of a platform (a user may watch a video of a given video available in an explorer setting in an applicable media player application; Fig 11, 12 and ¶ [0324]),
wherein each of the set of media items comprises a reference to one or more of a set of objects (a given video contains multiple relevant objects, such as objects 1104, 1105, 1108,1106, 1110; Fig 11, 12 and ¶ [0324], [0327]);
responsive to a request from a client device associated with a user of the platform for access to a media item of the set of media items (a user may select to watch a given user-initiated (selected) video in an applicable media player application; Fig 11, 12 and ¶ [0324], [0327]), determining one or more objects depicted within content of the media item that satisfy one or more criteria based on consumption data associated with the set of media items (a given frame of the video will contain multiple selectable elements, such as relevant objects 1104, 1105, 1108,1106, 1110, which are contextually relevant products and content recommendations (products algorithmically correlated based on data analytics of the audience and interaction with content ¶ [0092]-[0096], [0193]-[0198]); Fig 11, 12 and ¶ [0324], [0327]),
wherein the consumption data indicates whether a threshold number of other users of the platform have performed an action on a portion of at least one of the set of media items that depicts the one or more objects over a time period (contextually relevant products are identified based on in-depth analytics for audience insights (used by content publishers) to determine what content the audience is consuming, such as interactions with given products and may be based on user threshold levels of interactions; ¶ [0092]-[0096], [0193]-[0198]).
It would have been obvious to one skilled in the art at the time of the filing of the prior art that the in-depth analytics described in the cited prior art Boskovich would require (1) a quantifiable measure to determine a product as relevant, (2) that use of a threshold for determining user interactions with a product would identify relevant products for a given user, (3) that one of ordinary skill in the art would understand that a threshold number of users interacting with a product would identify analytics of products as relevant for a general audience, and (4) products are algorithmically correlated with high degree of interactions for increasing engagement opportunities thus making it obvious to try a threshold number of users interacting with a given product to determine the product as relevant and resulting with a reasonable expectation of success for determining the in-depth analytics (as related to consumption data). See MPEP § 2143(I)(E)).
Boskovich does not teach obtaining a source indicator associated with the determined one or more objects; and updating a user interface (UI) of the client device to include, with the media item, an annotation of the one or more objects to indicate that the one or more objects are associated with the source indicator.
Shihadah et al is analogous art pertinent to the technological problem addressed in the current application and teaches obtaining a source indicator associated with the determined one or more objects (frames of video is extracted 602, 604 with real objects depicted in the media content (¶ [0095]) in which logos, object identifiers, text are identified 606A, 606B, 606C are associated with merchandise/products the frame and may be matched 610 to additional content for the user 614; Fig 6 and ¶ [0089]-[0091]); and
updating a user interface (UI) of the client device to include, with the media item, an annotation of the one or more objects to indicate that the one or more objects are associated with the source indicator (if a product match is found based on the identification information 606, the user interface is updated for the user to select the product to purchase 614, which may be presented to the user with an annotation (annotation trigger button 1514 associated with the product and linked to a retailer website; Fig 6, 15C and ¶ [0089]-[0091], [0141]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Boskovich with Shihadah et al including to obtaining a source indicator associated with the determined one or more objects; and updating a user interface (UI) of the client device to include, with the media item, an annotation of the one or more objects to indicate that the one or more objects are associated with the source indicator. By using objects depicted with content of the media, marketers and users may quickly and efficiently identify objects quickly for product or service offers, resulting in further consumer options effectively and efficiently offered, as recognized by Shihadah et al (1 [0007]-[0009]).
Regarding Claim 2, Boskovich in view of Shihadah et al teach the method of claim 1 (as described above), wherein the request is received from the client device during a current time period, and wherein the time period indicated by the consumption data corresponds to at least one of the current time period, a prior time period that is prior to the current time period, or a future time period (Boskovich, a user-initiated video may be displayed to a user in a current time, while contextually relevant product and content recommendations based on in-depth analytics is obtained at a time prior to (before) the user watching the video (thereby providing the relevant in-scene product recommendations for the user); ¶ [0096], [0324], [0327]).
Regarding Claim 3, Boskovich in view of Shihadah et al teach the method of claim 1 (as described above), wherein the UI of the client device is updated to include the annotation in response to a detection of a user selection of the one or more objects via the UI (Boskovich, the user interface includes the annotation 1112 when the user hovers (or clicks to select) the mouse curser over the object of interest, which may then prompt additional relevant secondary content 1114 regarding the object; Fig 11 and ¶ [0325]-[0326]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Boskovich with Shihadah et al including the UI of the client device is updated to include the annotation in response to a detection of a user selection of the one or more objects via the UI. By providing the user’s display device to provide user-interactive elements, the user may be more engaged with the viewing content and may more easily access additional content and seamlessly, thereby improving user engagement, as recognized by Boskovich (¶ [0025]-[0026]).
Regarding Claim 4, Boskovich in view of Shihadah et al teach the method of claim 1 (as described above), wherein the set of media items comprises at least one of a video item or an audio item (Boskovich, media assets include videos or audio files; ¶ [0092], [0324]).
Regarding Claim 5, Boskovich in view of Shihadah et al teach the method of claim 1 (as described above), wherein the action indicated by the consumption data comprises one or more of a re- watching action, a pausing action, a rewinding action, a fast-forwarding action, or a zoom action (Boskovich, video content consumed by users to identify relevant product and content recommendations includes interactions based on a user clicking on the media content to play content, including to re-open (re-watch, further explore onscreen interests) and applied when a user initiates a pause function of the video; [0096], [0324]).
Regarding Claim 6, Boskovich in view of Shihadah et al teach the method of claim 1 (as described above), further comprising: performing digital image processing on a media item of the set of media items and recognizing the one or more of the set of objects based on the digital image processing (Boskovich, the system programming allows for interactive elements for the objects in the image frame 1112, with the interactive objects highlighted or contrasted for the user within the display 1114; Fig 11, 12 and ¶ [0324]-[0327]).
Regarding Claim 7, Boskovich in view of Shihadah et al teach the method of claim 1 (as described above), wherein updating the UI of the client device to include, with the media item, the annotation (Boskovich, interactive objects are annotated with being highlighted or contrasted for the user within the display 1114; Fig 11, 12 and ¶ [0325]-[0326]) comprises: updating the UI to include an emphasis of a region of an image frame of the media item that depicts the one or more objects (Boskovich, the display region of the image frame includes the interactive objects that are annotated for the user to be apprised of more interactive information; Fig 11, 12 and ¶ [0324]), wherein the emphasis comprises at least one of an outline, a highlight, a color altering, or a brightening of the region of the image frame (Boskovich, interactive objects are annotated with being highlighted or contrasted for the user within the display 1114; Fig 11, 12 and ¶ [0324]-[0326]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the current application to combine the teachings of Boskovich with Lewis et al including wherein updating the UI of the client device to include, with the media item, the annotation comprises: updating the UI to include an emphasis of a region of an image frame of the media item that depicts the one or more objects, wherein the emphasis comprises at least one of an outline, a highlight, a color altering, or a brightening of the region of the image frame. By providing the user’s display device to provide user-interactive elements, the user may be more engaged with the viewing content and may more easily access additional content and seamlessly, thereby improving user engagement, as recognized by Boskovich (¶ [0025]-[0026]).
Regarding Claim 8, Boskovich in view of Shihadah et al teach the method of claim 1 (as described above), wherein obtaining the source indicator associated with the determined one or more objects (Shihadah et al, frames of video is extracted 602, 604 with real objects and identifiers 606A, 606B, 606C are identified in the frame; Fig 6 and ¶ [0089]) comprises: determining a plurality of sources that are associated with the one or more objects (Shihadah et al, different source indicators (logos, object identifiers, text are identified 606A, 606B, 606C) are associated with merchandise/products the frame; Fig 6 and ¶ [0089]-[0091], [0141]); and selecting a source from the plurality of sources based on contextual data associated with the client device, wherein the source indicator corresponds to the selected source (Shihadah et al, an analysis is performed 608 to identify a match between a given promotional advertisement (source identifier) data and the given product in the frame; Fig 6 and ¶ [0089]-[0090]).
Regarding Claim 9, Boskovich in view of Shihadah et al teach the method of claim 8 (as described above), wherein the contextual data comprises at least one of a geographic location of the client device, a source preference associated with the client device, or an availability of the one or more objects (Shihadah et al, the promotional advertisement data identified in the frame is analyzed to determine if a match is found 608, which includes if the match is found and the product is available to purchase 614; Fig 6 and ¶ [0090]-[0091]).
Regarding Claim 10, Boskovich teach a system (server 140; Fig 1, 2 and ¶ [0136]-[0137]); and a processing device coupled to the memory (processing device 202 coupled to memory 204; Fig 2 and ¶ [0140]-[0141]), the processing device to perform operations (processor 202 executes program stored in memory 204; Fig 2 and ¶ [0141]) comprising: steps identical to claim 1 (as described above).
Regarding Claim 11, Boskovich in view of Shihadah et al teach the system of claim 10 (as described above), wherein further limitations are identical to claim 2 (as described above).
Regarding Claim 12, Boskovich in view of Shihadah et al teach the system of claim 10 (as described above), wherein further limitations are identical to claim 3 (as described above).
Regarding Claim 13, Boskovich in view of Shihadah et al teach the system of claim 10 (as described above), wherein further limitations are identical to claim 4 (as described above).
Regarding Claim 14, Boskovich in view of Shihadah et al teach the system of claim 10 (as described above), wherein further limitations are identical to claim 5 (as described above).
Regarding Claim 15, Boskovich in view of Shihadah et al teach the system of claim 10 (as described above), wherein further limitations are identical to claim 6 (as described above).
Regarding Claim 16, Boskovich in view of Shihadah et al teach the system of claim 10 (as described above), wherein further limitations are identical to claim 7 (as described above).
Regarding Claim 17, Boskovich teach a non-transitory computer readable storage medium comprising instructions (memory 204 contains one or more computer-executable instructions; Fig 2 and ¶ [0140]-[0141]) that, when executed by a processing device, cause the processing device to perform operations (processor 202 executes program stored in memory 204; Fig 2 and ¶ [0140]-[0141]) comprising: steps identical to claim 1 (as described above).
Regarding Claim 18, Boskovich in view of Shihadah et al teach the non-transitory computer readable storage medium of claim 17 (as described above), wherein further limitations are identical to claim 2 (as described above).
Regarding Claim 19, Boskovich in view of Shihadah et al teach the non-transitory computer readable storage medium of claim 17 (as described above), wherein further limitations are identical to claim 3 (as described above).
Regarding Claim 20, Boskovich in view of Shihadah et al teach the non-transitory computer readable storage medium of claim 17 (as described above), wherein further limitations are identical to claim 4 (as described above).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Scott-Green et al (US 2022/0027624, application 16/627,991, originally cited in Non-Final Rejection 10/27/2025), by the same inventors and applicant, discloses annotating and source linking objects displayed in an image, with claim limitations focused on selecting objects based viewership data and associating the selected object with a source indicator.
Grossman et al (US 2018/0121470, originally cited in Non-Final Rejection 10/27/2025) teach a system and method for acquiring and sharing annotations of objects identified in images with annotations stored in a database and linked to pre-defined object data.
Van Zwol et al (US 2018/0047064, originally cited in Non-Final Rejection 10/27/2025) teach a system and method for contextual media enrichment presentation items of media objects identified in an image and searched for similar object identifiers over the internet.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATHLEEN M BROUGHTON whose telephone number is (571)270-7380. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATHLEEN M BROUGHTON/Primary Examiner, Art Unit 2661
Prosecution Insights