DETAILED ACTION
I. Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
II. Restriction
Applicant’s election without traverse of Species II in the reply filed on July 10, 2026 is acknowledged. Accordingly, claims 2-8 are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected species, there being no allowable generic or linking claim.
III. Priority/35 U.S.C. 112(a)
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 120 is acknowledged. However, Applicant has not complied with one or more conditions for receiving the benefit of an earlier filing date under 35 U.S.C. 120. A later-filed application must be an application for a patent for an invention which is also disclosed in the prior application (the parent or original nonprovisional application or provisional application). The disclosure of the invention in the parent application and in the later-filed application must be sufficient to comply with the requirements of 35 U.S.C. 112(a), except for the best mode requirement. Transco Products, Inc. v. Performance Contracting, Inc., 38 F.3d 551, 32 USPQ2d 1077 (Fed. Cir. 1994).
The disclosure of the prior-filed applications, 18/099,196, 16/214,112, 15/973,34915/190,140, 14/491,863, 13/908,900, fail to provide adequate support in the manner provided by 35 U.S.C. 112(a) for claims 12,13,15 and 18-20 of this application. Specifically, Applicant’s alternate use of a mobile device that captures an image of a display screen, accepts user selection of an object in the captured image, and accesses information corresponding to the selected object is directed to an embodiment that is either inconsistent with the disclosed use of meta data to access object information or whose operation with the use of meta data does not appear to be possessed by Applicant as of the instant application’s effective filing date.
A. Claims 12,13, and 18-20: meta data
Claims 12,13, and 18-20 each disclose using meta data associated with an object in the image to access the object information, where the meta data comprises a URL/internet address of a remote database or coordinates of the object on a screen of a separate device. However, the disclosed passages addressing the use of associated meta data support its use when a remote control provides cursor control on the video displayed on a separate device display (see specification of parent application 13/908,900, p. 21, line 27 – p. 21, line 4). The examiner recognizes that, later on p. 22 of the ‘900 specification, Applicant discloses that “a user may capture the image on a device 110 such as a smart phone or tablet and identify product coordinates by a touch interface or stylus and simply touching or encircling the product on the captured image” and that “[s]uch alternate approaches are similarly within the scope of this aspect of the invention associating display image coordinates, or range of coordinates, with product placement meta data, identifying a product location in an image by a user and employing this for retrieval of product information.” However, the details that require this alternate approach of mobile device operation with the use of meta data are not disclosed with enough specificity to conclude that Applicant possessed the full scope of the claims 12,13, and 18-20.
In the case of the remote control providing cursor control disclosed on p. 21, line 27 – p. 21, line 4 of the ‘196 specification, user selection of an object on a device display can be detected by the content provider via a set-top box or the like and, therefore, meta data may be accessed and employed to provide the object information. However, when a mobile device that does not have that same connection to the set-top box is used to capture an image of the display device, accept user selection of an object in the captured image, and access object information using meta data, operation is murkier. That is, the specification does not elaborate on the subsequent steps that are performed after the capture of the device display image by the mobile device. Presumably, that image would be transmitted to a remote database that identifies the selected object and fetches object information that is then sent to the mobile device. If, however, meta data is accessed locally by the mobile device, the specification is not clear how the mobile device can access meta data that is sent along with the image by the content provider through the mere capture of the image on the device display. Applicant discloses an embodiment in which the mobile device displays content synchronized with that on a display (i.e., television display or the like). In this case, it is understandable how the mobile device can then access the meta data with the image as it is a part of the synchronized content. However, it is less understandable why capture of the display would be necessary.
Applicant also discloses, on p. lines 28-30 of the ‘900 specification, that the meta data
may be kept in a remote product placement database. In that case, if the mobile device transmits the captured image with the selected object to the remote database, how that stored meta data is used to fetch object information is unclear. Also, claims 13 and 20 require that the meta data is a URL of the remote database and an internet address. The examiner finds it unlikely that the captured image with the selected object is used by the remote database to access meta data. Rather, Applicant more likely envisioned that the meta data is accessed locally, and the URL/internet address provides the location of the stored object information.
All this is not to say that one of ordinary skill could not design a system that incorporates a mobile device that captures an image of a display screen, accepts user selection of an object in the captured image, and accesses information corresponding to the selected object by employing meta data. Rather, the lack of detail regarding the mobile device’s access to and use of meta data and the need for the examiner to fill in so many blanks compels the conclusion that Applicant did not fully conceive of the design. Indeed, on p. 22, lines 15-17, Applicant expressly discloses that the mobile device’s access to and use of meta data is within the scope of disclosure. However, the examiner sees this as a catch-all statement that lacks appreciation for the nuance required for that operation.
B. Claim 15: reference markers
Claim 15 requires that the captured image comprises “one or more reference patterns which are employed to identify a remote database having the object information.” Again, Applicant only discloses this in context of a remote control proving cursor control when the selection by the user can be recognized by a set-top box providing the content. The specification does not flesh out the details of how the reference markers and/or the selected object are used together to access the object information.
IV. Claim Rejections - 35 U.S.C. § 112
The following is a quotation of 35 U.S.C. 112:
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 16 is rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention. Specifically, on lines 1 and 2, the claim requires “the image displayed on the screen of a separate device” for which the claim lack antecedent basis. Claim 1 establishes an image displayed on a display of a first device. The examiner believes that the separate device is the first device and would suggest a corresponding amendment to claim 16.
V. Claim Rejections - 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 9-11,14,16,17, and 21-28 are rejected under 35 U.S.C. 103 as being unpatentable over Ritzau et al. (US # 8,165,409 B2).
As to claim 9, Ritzau et al. teaches an interactive method (Fig. 8), comprising:
displaying on a display of a first device an image including one or more distinct object images (col. 17, lines 3-10; note also, col. 13, lines 55-59; {Ritzau’s media information obtaining applies to capture of virtually any image or video, including that which is displayed on a television display, for example.});
capturing image information in a freely movable second device (Fig. 2, device “200”; col. 3, lines 9 and 10) including a camera (Fig. 2, camera “270”) and a screen (Fig. 2, display “230”), the image information corresponding to a scene in the field of view of the camera captured by the camera which includes at least part of the display of the first device including one or more of the object images (col. 17, lines 3-10; {Again, Ritzau’s media information obtaining applies to capture of virtually any image or video, including that which is displayed on a television display.});
displaying on the screen of the second device an image corresponding to the captured scene including one or more of the object images (e.g., Figa. 6A and 6B; col. 6, lines 42 and 43, noting also col. 17, lines 15-18);
detecting a selection by a user within a range of coordinates generally corresponding to a displayed object image (col. 17, lines 18-21); and
accessing information corresponding to the selected object (e.g., Fig. 4B; col. 17, lines 21-25).
Ritzau et al. discloses that a user may mark a face (or other object in a captured image) using any of number of options, including an icon, but does not expressly disclose that the device display “230” is a touch screen that accepts user input, whereby a user can mark a face or object by touching the position on a touch screen. However, the examiner takes official notice to the use of a touch screen as the display on a mobile device, like Ritzau’s, as well known in the art. One of ordinary skill in the art would have been motivated to design Ritzau’s display as a touch screen that accepts touch input for the selection of the face or object because touch screens minimize the size of the host device by eliminating extraneous input hardware and present selection in a more intuitive, user-friendly manner.
As to claim 10, Ritzau et al. teaches the method of claim 9, wherein the second device is a smartphone (Fig. 2; col. 4, lines 21 and 22; {The examiner submits that a cellular telephone with Ritzau’s functionality qualifies it as a smartphone.}) or tablet.
As to claim 11, Ritzau et al. teaches the method of claim 9, wherein detecting a touch selection by a user of a displayed object image comprises detecting the user touching the object image displayed on the touch screen or the user generally encircling the object image displayed on the touch screen (col. 17, lines 18-21).
As to claim 14, Ritzau et al. teaches the method of claim 9, wherein accessing information corresponding to the selected object comprises accessing a remote database containing object image identification information to identify the object (e.g., col. 7, lines 5-11).
As to claim 16, Ritzau et al. teaches the method of claim 15, wherein the image displayed on a screen ofthe first device comprises a video and wherein the displayed object is a product within the video and wherein the additional application processing comprises accessing product information at the remote database (col. 13, lines 55-59).
As to claim 17, Ritzau et al. teaches an interactive system (Fig. 2, device “200” interacting with the database mentioned in col. 7, lines 5-11 or interacting with captured subject) having an improved user interface (e.g., Figs. 6A and 6B), comprising:
a freely movable device (Fig. 2, device “200”) including a camera (Fig. 2, camera “270”), a screen (Fig. 2, display “230”), and one or more processors (Fig. 3, processing logic “310”; col. 5, lines 23-26) with associated programmed instructions (col. 5, lines 26-31) configured in volatile or nonvolatile storage (Fig. 3, storage “320”), wherein the device camera is responsive to a user control to capture a scene in the field of view of the camera (col. 10, lines 62-67) including one or more distinct user identifiable objects (e.g., col. 13, lines 55-59) and display on the screen an image including one or more of the object images (col. 11, lines 9-18);
wherein the programmed instructions are responsive to a user input at coordinates within a range of coordinates generally corresponding to the displayed object image (col. 11, lines 19-29) to initiate control processing to access information corresponding to the selected object (e.g., Fig, 4B; col. 7, lines 5-11).
Ritzau et al. discloses that a user may mark a face (or other object in a captured image) using any of number of options, including an icon, but does not expressly disclose that the device display “230” is a touch screen that accepts user input, whereby a user can mark a face or object by touching the position on a touch screen. However, the examiner takes official notice to the use of a touch screen as the display on a mobile device, like Ritzau’s, as well known in the art. One of ordinary skill in the art would have been motivated to design Ritzau’s display as a touch screen that accepts touch input for the selection of the face or object because touch screens minimize the size of the host device by eliminating extraneous input hardware and present selection in a more intuitive, user-friendly manner.
As to claim 21, Ritzau et al. teaches the interactive system of claim 17, wherein detecting a touch selection by a user of a displayed object image comprises detecting the user touching the object image displayed on the touch screen or the user generally encircling the object image displayed on the touch screen (col. 11, lines 24-26).
As to claim 22, Ritzau et al. teaches the interactive system of claim 17, wherein detecting a touch input by a user within a range of coordinates generally corresponding to the displayed object image comprises detecting the user touching the touch screen and generally encircling the object image displayed on the touch screen (Fig. 6B, box “670”; col. 11, lines 24-26; {The examiners reads a box as a general encircling.}).
As to claim 23, Ritzau et al. teaches an interactive method (Fig. 8) providing a dynamic user interface (Figs. 6A and 6B), comprising:
capturing image information in a freely movable device (Fig. 2, device “200”) having a camera (Fig. 2, camera “270”) and a screen (Fig. 2, display “230”), the image information corresponding to a scene in the field of view of the camera captured by the camera (col. 17, lines 3-10);
displaying on the screen of the device an image corresponding to the captured scene including one or more object images (Figs. 6A and 6B);
detecting a selection by a user within a range of coordinates generally corresponding to a displayed object image (col. 17, lines 18-21); and
accessing information corresponding to the selected object (e.g., Fig. 4B; col. 7, lines 5-11).
Ritzau et al. discloses that a user may mark a face (or other object in a captured image) using any of number of options, including an icon, but does not expressly disclose that the device display “230” is a touch screen that accepts user input, whereby a user can mark a face or object by touching the position on a touch screen. However, the examiner takes official notice to the use of a touch screen as the display on a mobile device, like Ritzau’s, as well known in the art. One of ordinary skill in the art would have been motivated to design Ritzau’s display as a touch screen that accepts touch input for the selection of the face or object because touch screens minimize the size of the host device by eliminating extraneous input hardware and present selection in a more intuitive, user-friendly manner.
As to claim 24, Ritzau et al. teaches the interactive method of claim 23, wherein detecting a touch input by a user within a range of coordinates generally corresponding to the displayed object image comprises detecting the user touching the touch screen and generally encircling the object image displayed on the touch screen (Fig. 6B, box “670”; col. 17, lines 18-21; {The examiners reads a box as a general encircling.}).
As to claim 25, Ritzau et al. teaches the interactive method of claim 23, wherein detecting a touch input by a user within a range of coordinates generally corresponding to the displayed object image comprises detecting the user touching the object image displayed on the touch screen (Fig. 4A, cursor “130”; col. 17, lines 18-21).
As to claim 26, Ritzau et al. teaches interactive method of claim 23, wherein accessing information corresponding to the selected object comprises accessing a remote database via the internet (col. 5, lines 58-63; col. 6, lines 55-59).
As to claim 27, Ritzau et al. teaches the interactive method of claim 23, wherein accessing information corresponding to the selected object comprises accessing a remote database containing object image identification information to identify the object (col. 5, lines 5-11 and lines 58-63).
As to claim 28, Ritzau et al. teaches the interactive method of claim 23, wherein the freely movable device is a smartphone (Fig. 2; col. 4, lines 21 and 22; {The examiner submits that a cellular telephone with Ritzau’s functionality qualifies it as a smartphone.}).
VI. Additional Pertinent Prior Art
Stafford et al. (US # 8,730,354 B2), Ortiz et al. (US 2012/0167001 A1), and Roberts et al. (US 2012/0120296 A1) each disclose additional examples of an interactive system in which a user captures an image of a display including objects for which the user can access additional information, Stafford et al. specifically disclosing the use of markers on the displayed content to facilitate information retrieval and Roberts et al. disclosing the use of metadata to associate the objects with the information.
VII. Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J DANIELS whose telephone number is (571)272-7362. The examiner can normally be reached M-F 9:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached at 571-272-7564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J DANIELS/Primary Examiner, Art Unit 2637
9/1/2026
Prosecution Insights