DETAILED ACTION
This Office Action is responsive to the Applicant’s submission, filed on November 6, 2025, amending claims 1-4, 6-10 and 12-18. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on January 26, 2026 has been considered by the Examiner.
Claim Objections
Claims 1-19 are objected to because of the following informalities. Appropriate correction is required.
Claim 1 recites both a “first software plugin” and “first software plug-in.” To avoid any potential reader confusion, the spelling of “plug-in” should be consistent. The same rationale applies to a “second software plugin” and “second software plug-in” also recited in claim 1.
Also in claim 1, there is no antecedent basis for “the one or more images of the identification document” recited therein. Prior to this phrase, claim 1 recites “one or more images” and “an identification document,” but not one or more images of such an identification document.
Also in claim 1, there is no antecedent basis for “the second recognized information” recited therein. Prior to this phrase, claim 1 recites “generate a second recognized input information from the second input information,” but not “second recognized information” per se.
Claims 2-6 and 19 depend from claim 1 and thereby include all of the limitations of claim 1. Accordingly, claims 2-6 and 19 are also objected to under the same rationale as described above with respect to claim 1.
Claim 7 recites both a “first software plugin” and “first software plug-in.” To avoid any potential reader confusion, the spelling of “plug-in” should be consistent. The same rationale applies to a “second software plugin” and “second software plug-in” also recited in claim 7.
Also in claim 7, there is no antecedent basis for “the one or more images of the identification document” recited therein. Prior to this phrase, claim 7 recites “one or more images” and “an identification document,” but not one or more images of such an identification document.
Also in claim 7, there comprises a typographical error (i.e. missing terms) within the phrase, “the first recognized input information and the second recognized obtained from each of the ….”
Also in claim 7, there is no antecedent basis for “the combined first recognized input information and second recognized information” recited therein. In particular, prior to this phrase, claim 7 recites “second recognized input information” but not “second recognized information” per se.
Claims 8-12 depend from claim 8 and thereby include all of the limitations of claim 8. Accordingly, claims 8-12 are also objected for the same reasons as described above for claim 8.
Claim 13 recites both a “first software plugin” and “first software plug-in.” To avoid any potential reader confusion, the spelling of “plug-in” should be consistent. The same rationale applies to a “second software plugin” and “second software plug-in” also recited in claim 13.
Also in claim 13, there is no antecedent basis for “the one or more images of the identification document” recited therein. Prior to this phrase, claim 13 recites “one or more images” and “an identification document,” but not one or more images of such an identification document.
Also in claim 13, there is no antecedent basis for “the second recognized information” recited therein. Prior to this phrase, claim 13 recites “generate a second recognized input information from the second input information,” but not “second recognized information” per se.
Claims 14-18 depend from claim 13 and thereby include all of the limitations of claim 13. Accordingly, claims 14-18 are objected to under the same rationale as described above with respect to claim 13.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-19 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent No. 10,572,215 to Cooper et al. (“Cooper”), and also over U.S. Patent Application Publication No. 2015/0073907 to Purves et al. (“Purves”).
Regarding claim 1, Cooper describes an augmented reality (AR) system in which a computing device is enabled to analyze information, such as sensor data captured by one or more sensors of the computing device, to identify one or more objects represented in the information and to obtain and present supplemental content based on the identified objects (see e.g. column 1, line 62 – column 2, line 41). Like claimed, Cooper particularly teaches that the computing device can be a mobile computing device (see e.g. column 12, lines 24-38) comprising:
an image sensor configured to capture one or more images from a field of view of the image sensor (see e.g. column 2, lines 21-26; column 2, line 56 – column 3, line 10; column 8, lines 9-24; and column 12, lines 24-38: Cooper discloses that the mobile computing device can comprise one or more sensors, including a camera for acquiring images and/or video from a field of view of the camera. The camera is considered an image sensor like claimed.);
a visual display (see e.g. column 3, lines 6-12; column 8, lines 9-24; and column 13, lines 30-37: Cooper discloses that the mobile computing device comprises a display screen. Such a display screen is considered a visual display like claimed.);
a memory for storing a software program comprising a plurality of software plug-ins (see e.g. column 13, lines 50-60: Cooper discloses that the mobile computing device includes memory and storage for storing processor-executable instructions. Cooper discloses that the processing of the AR system can be implemented via an application executed on the mobile computing device – see e.g. column 16, lines 36-45. Cooper further discloses that the AR system comprises an extensible architecture in which users can dynamically install and/or activate plug-ins to enable the identification of new types of objects or supplemental content relating to objects – see e.g. column 2, lines 4-9; column 4, lines 20-43; and column 16, lines 6-35. Cooper teaches that the plug-ins can be partially or entirely implemented on the mobile computing device – see e.g. column 19, line 22-56; column 21, lines 6-46; column 23, line 19 – column 24, line 10; and column 26, lines 4-36. In such embodiments, the memory/storage of the mobile computing device would necessarily comprise a software program comprising a plurality of software plug-ins, i.e. the AR application and its plug-ins.); and
a processor (see e.g. column 13, lines 50-60: Cooper discloses that the mobile computing device comprises a processor/controller for executing processor-executable instructions), the processor configured to:
display, in real time, one or more images captured by the image sensor on the visual display (see e.g. column 3, lines 6-15; and column 8, lines 9-24: Cooper discloses that the display screen of the mobile computing device can display a “live” view of image frames captured by the camera. Cooper further discloses that the augmented reality system operates in real-time to provide the user with the supplemental content and/or functionality while engaging with the primary content captured by the sensors – see e.g. column 2, lines 32-41.);
select one or more software plug-ins of the plurality of software plug-ins that is capable of recognizing one or more input information in the one or more images, wherein the one or more software plug-ins of the plurality of software plug-ins is capable of recognizing information contained within a document (see e.g. column 2, lines 11-32; and column 8, lines 9-61: like noted above, Cooper teaches that the mobile computing device is enabled to analyze captured sensor information to identify one or more objects represented therein. Cooper particularly discloses that a recognition module performs recognition on the content captured by the mobile device to identify one or more objects therein – see e.g. column 10, line 65 – column 11, line 15; and column 17, lines 5-37. Moreover, Cooper teaches that the augmented reality system can be extended via one or more plug-ins, e.g. “matchers,” to identify particular objects within the sensor data – see e.g. column 16, lines 6-22; column 17, lines 17-37; column 20, line 60 – column 21, line 5; and column 26, lines 4-36. The plug-ins employed to recognize information can be selected automatically or by a user – see e.g. column 4, lines 34-43, column 21, lines 27-46; and column 23, line 43 – column 24, line 7. Cooper further teaches that such functionality, including the plug-ins, can be partially or entirely implemented on the mobile computing device – see e.g. column 4, lines 2-6; column 9, lines 15-24; column 10, lines 5-11; column 14, lines 12-34; column 21, lines 6-46; and column 26, lines 4-36. In such embodiments, the processor selects one or more software plug-ins of the plurality of plug-ins to recognize one or more input information, i.e. one or more objects, in the images captured by the camera of the mobile computing device. Cooper suggests that the one or more software plug-ins are particularly capable of recognizing objects contained within a document, e.g., to recognize text, phone numbers and/or web addresses within an image of a document captured by the camera – see e.g. column 1, line 62 – column 2, line 10; column 2, line 56 – column 3, line 28; column 4, lines 20-43; column 5, line 48 – column 6, line 14; column 7, lines 30-63; and FIGS. 2A-D.);
input one or more images of the document to each of the selected one or more software plug-ins (see e.g. column 16, lines 6-22; column 17, lines 17-37; column 20, line 60 – column 21, line 5; and column 26, lines 4-36: like noted above, Cooper teaches that the augmented reality system can be extended via one or more plug-ins, e.g. “matchers,” to identify particular objects within the sensor data. To recognize the particular objects, sensor data such as images are provided to each of the plug-ins/matchers – see e.g. column 17, lines 5-37; column 21, lines 6-60; column 22, lines 4-25; and column 26, lines 4-36. As noted above, Cooper suggests that the one or more software plug-ins/matchers are capable of recognizing objects contained within a document, and so it is apparent that in such embodiments one or more images of the document are input to each of the one or more matchers/plug-ins to recognize objects therein.);
utilize a first software plug-in of the one or more software plug-ins to recognize a first input information contained within the document from an image of the document, and utilize a second software plug-in of the one or more software plug-ins to recognize a second input information contained within the document from an image of the document, wherein the first software plug-in is different from the second software plug-in and wherein the first input information is different from the second input information (see e.g. column 17, line 5 – column 18, line 5; column 21, lines 6-38; column 22, lines 4-25; and column 26, lines 4-36: Cooper teaches that the sensor data can be provided to multiple plug-ins/matchers in parallel, wherein each plug-in/matcher is configured to recognize particular objects within the sensor data, and if a matcher/plug-in recognizes an object within the sensor data, it returns a digital entity with facet data indicating the recognized object. Cooper further suggests that multiple objects can be recognized within a document from the image of the document utilizing multiple respective plug-ins – see e.g. column 1, line 62 – column 2, line 10; column 2, line 56 – column 3, line 28; column 4, lines 20-43; column 5, line 48 – column 6, line 14; column 7, lines 30-63; and FIGS. 2A-D. In such embodiments, a first plug-in/matcher is utilized to recognize first input information, i.e. a first object, contained within the document from an image of the document, and a second plug-in/matcher is utilized to recognize a second object contained within the document from an image of the document, wherein the first plug-in/matcher is different from the second plug-in/matcher and wherein the first object recognized by the first plug-in/matcher is different from the second object recognized by the second plug-in/matcher.);
generate first recognized input information from the first input information, and generate second recognized input information from the second input information (see e.g. column 17, line 5 – column 18, line 5; column 21, lines 6-38; column 22, lines 4-25; and column 26, lines 4-36: like noted above, Cooper teaches that the sensor data can be provided to multiple plug-ins/matchers in parallel, wherein each plug-in/matcher is configured to recognize particular objects within the sensor data, and if a plug-in/matcher recognizes an object within the sensor data, it returns a digital entity with facet data indicating the recognized object. Cooper further suggests that multiple objects can be recognized within a document from the image of the document utilizing multiple respective plug-ins – see e.g. column 1, line 62 – column 2, line 10; column 2, line 56 – column 3, line 28; column 4, lines 20-43; column 5, line 48 – column 6, line 14; column 7, lines 30-63; and FIGS. 2A-D. In such embodiments where multiple objects, e.g. first and second input information, are recognized in a document by multiple respective plug-ins/matchers, a first digital entity with facet data indicating a first recognized object would be generated by a first plug-in/matcher, and a second digital entity with facet data indicating a second recognized object would be generated by a second plug-in/matcher. The first digital entity with facet data indicating the first recognized object is considered first recognized input information generated from the first input information, and the second digital entity with facet data indicating the second recognized object is considered second recognized input information generated from the second input information like claimed.);
combine the first recognized input information and the second recognized information obtained from the first software plug-in and the second software plug-in (As described above, Cooper teaches embodiments where multiple objects are recognized in a document by multiple respective plug-ins/matchers, wherein a first digital entity with facet data indicating a first recognized object is generated by a first plug-in/matcher, and a second digital entity with facet data indicating a second recognized object is generated by a second plug-in/matcher; the first digital entity with facet data indicating the first recognized object is considered first recognized input information, and the second digital entity with facet data indicating the second recognized object is considered second recognized input information. Cooper suggests that the digital entity and facet data are stored in memory, e.g. so that “resolvers” can identify supplemental content and/or functions corresponding to the entities – see e.g. column 17, line 34 – column 18, line 45; and column 26, lines 19-63. Accordingly, in embodiments where multiple objects are recognized in a document by multiple respective plug-ins/matchers, the entity and facet data provided by the different plug-ins/matchers would understandably be stored in memory. This can be considered combining, e.g. into the same memory, the entity and facet data provided by the different plug-ins/matchers. Cooper thus further teaches combining the first recognized input information and the second recognized input information, i.e. the first and second entity and facet data, obtained from the first plug-in/matcher and the second plug-in/matcher.);
convert the combined first recognized input information and second recognized information into one or more contextual messages (As described above, Cooper teaches embodiments where multiple objects are recognized in a document by multiple respective plug-ins/matchers, wherein a first digital entity with facet data indicating a first recognized object is generated by a first plug-in/matcher, and a second digital entity with facet data indicating a second recognized object is generated by a second plug-in/matcher. Like further described above, Cooper suggests that the digital entity and facet data are stored in memory, e.g. so that “resolvers” can identify supplemental content and/or functions corresponding to the entities – see e.g. column 17, line 34 – column 18, line 45; and column 26, lines 19-63. Such supplemental content and/or functions can be considered contextual messages like claimed. Accordingly, Cooper is considered to teach converting the combined first recognized input information and second recognized input information, i.e. the first and second entity and facet data stored in memory, into one or more contextual messages, i.e. into supplemental content and/or functions.); and
present the one or more contextual messages, corresponding to the first software plug-in and the second software plug-in, on the visual display (see e.g. column 2, lines 32-41: Cooper discloses that the augmented reality system operates in real-time to provide the user with the supplemental content while engaging with the primary content captured by the sensors. Like noted above, Cooper discloses that the supplemental content can comprise information and/or selectable functions associated with each object identified in the sensor information – see e.g. column 6, lines 15-57; column 8, lines 24-61; and column 18, line 46 – column 19, line 3. Cooper particularly demonstrates that the display screen of the mobile device presents the supplemental content and/or selectable functions, i.e. the one or more contextual messages corresponding to the first software plug-in/matcher and the second software plug-in/matcher, via “ribbons” overlaid on the image frames captured by the camera of the mobile device – see e.g. column 5, line 48 – column 6, line 57; column 7, lines 30-63; column 18, lines 46-66; column 26, lines 37-63; and FIGS. 2A-D.).
Accordingly, Cooper teaches a mobile computing device similar to that of claim 1, but does not explicitly disclose that the document is an identification document, and wherein the software plug-ins are capable of verifying the one or more input information contained within the identification document, as is required by claim 1.
Purves nevertheless describes a mobile computing device that (i) displays, in real-time, one or more images captured by the device, (ii) recognizes input information in the one or more images including e.g. by verifying information contained within an identification document (e.g. within a payment card, a license, a library membership card, or an insurance card), and (iii) presents one or more contextual messages (e.g. overlay labels) related to the recognized input information (see e.g. paragraphs 0205, 0209-0210, 0216, 0218-0222 and 0250-0252, and FIGS. 9A-B and 11-13).
It would have been obvious to one of ordinary skill in the art, having the teachings of Cooper and Purves before the effective filing date of the claimed invention, to modify the software plug-ins taught by Cooper, which recognize input information in a document, so as to particularly be applicable to an identification document like taught by Purves, wherein the one or more software plug-ins would be capable of recognizing and verifying information contained within the identification document. It would have been advantageous to one of ordinary skill to utilize such a combination because it would enable relevant contextual messages to be presented to the user regarding the identification document, as is demonstrated by Purves (see e.g. paragraphs 0209-0210, 0218-0222 and 0252, and FIGS. 9A-B and 11-13). Accordingly, Cooper and Purves are considered to teach, to one of ordinary skill in the art, a mobile computing device like that of claim 1.
As per claim 2, Cooper further teaches that the one or more input information comprises at least one of a recognizable object or a symbol (see e.g. column 2, lines 11-32; and column 8, lines 9-61: like noted above, Cooper teaches that mobile computing device is enabled to analyze captured sensor information to identify one or more objects represented therein.). Purves provides a similar teaching (see e.g. paragraphs 0209-0210, 0216, 0218-0222 and 0250-0252, and FIGS. 9A-B and 11-13). Accordingly, the above-described combination of Cooper and Purves further teaches a mobile computing device like that of claim 2.
As per claim 3, Cooper further teaches that the first software plug-in and the second software plug-in utilized by the processor are based on user input, and wherein the user input is indicative of user behavior (see e.g. column 23, lines 43-63: Cooper discloses that certain plug-ins can be dynamically activated based on user behavior, e.g. when the user leaves his or her home. Additionally, Cooper discloses that specific plug-ins for certain sensor data can be selected by the user – see e.g. column 21, lines 27-37. Cooper also discloses that the identification of objects within the sensor data, e.g. by the plug-ins, can be performed in response to user input indicative of user behavior, e.g. in response to gestures – see e.g. column 5, lines 35-47.). Purves similarly teaches that the identification of objects within the sensor data can be performed in response to user input indicative of user behavior (e.g. pointing gestures) (see paragraphs 0209-0210, 0216, 0218-0222 and 0250-0252). Like noted above, Cooper teaches that such recognition can be performed by one or more plug-ins (see e.g. column 16, lines 6-22; column 17, lines 17-37; column 20, line 60 – column 21, line 5; and column 26, lines 4-36). Accordingly, the above-described combination of Cooper and Purves further teaches a mobile computing device like that of claim 3, in which the first software plug-in and the second software plug-in utilized by the processor are based on user input, and wherein the user input is indicative of user behavior.
As per claim 4, Cooper further teaches that the processor is configured to utilize the first software plug-in and the second software plug-in to generate one or more information elements from the one or more input information (see e.g. column 16, lines 6-22; column 17, lines 17-37; column 20, line 60 – column 21, line 5; and column 26, lines 4-36: like noted above, Cooper teaches that one or more plug-ins, e.g. “matchers,” can be utilized to identify particular objects within the sensor data. The processor thus executes the one or more plug-ins, i.e. the first software plug-in and the second software plug-in, to generate one or more information elements from the input information, i.e. to generate indications of identified objects in the sensor data.). Accordingly, the above-described combination of Cooper and Purves further teaches a mobile computing device like that of claim 4.
As per claim 5, Cooper teaches that the processor is configured to convert the one or more information elements to the one or more contextual messages (see e.g. column 11, line 16 – column 12, line 23: Cooper discloses that a supplemental content module and a function module operate to identify supplemental content and functions related to the objects identified in the sensor content. Cooper also teaches that one or more resolver plug-ins can be utilized to provide the supplemental content and/or functions associated with the identified objects – see e.g. column 16, lines 22-35; column 18, lines 6-45; column 19, lines 22-56; and column 26, lines 37-63. The processor thus executes the supplemental content module, the function module and/or different plug-ins to convert the one or more information elements, i.e. the identified one or more objects, into the one or more contextual messages, i.e. supplemental content and/or functions.). Accordingly, the above-described combination of Cooper and Purves further teaches a mobile computing device like that of claim 5.
As per claim 6, Cooper further teaches decoding a symbol to generate the one or more information elements in an instance in which the one or more input information includes the symbol, wherein the symbol corresponds to a barcode (see e.g. column 2, lines 21-41; column 6, lines 26-39; column 17, line 65 – column 18, line 5; and column 18, lines 22-32: Cooper discloses that the objects recognized by the AR system can include bar codes, whereby the supplemental information and/or functions relate to the object. In such circumstances, the AR system understandably decodes a symbol to generate one or more information elements, wherein the symbol corresponds to a bar code.). Accordingly, the above-described combination of Cooper and Purves further teaches a mobile computing device like that of claim 6.
Regarding claims 7 and 13, Cooper describes an augmented reality (AR) system in which a computing device is enabled to analyze information, such as sensor data captured by one or more sensors of the computing device, to identify one or more objects represented in the information and to obtain and present supplemental content based on the identified objects (see e.g. column 1, line 62 – column 2, line 41). Like claimed, Cooper particularly teaches:
receiving, by a processor, one or more images of a field of view of an image sensor (see e.g. column 2, lines 21-26; column 2, line 56 – column 3, line 10; column 8, lines 9-24; and column 12, lines 24-38: Cooper describes a mobile computing device that comprises one or more sensors, including a camera for acquiring images and/or video from a field of view of the camera. Cooper further discloses that the mobile computing device also comprises a processor/controller for executing processor-executable instructions for gathering and processing the sensor data – see e.g. column 13, lines 50-65.);
displaying, by the processor in real-time, the one or more images captured by the image sensor (see e.g. column 3, lines 6-15; and column 8, lines 9-24: Cooper discloses that the mobile computing device comprises a display screen that can display a “live” view of image frames captured by the camera. Cooper further discloses that the augmented reality system operates in real-time to provide the user with the supplemental content and/or functionality while engaging with the primary content captured by the sensors – see e.g. column 2, lines 32-41.);
selecting, by the processor, one or more software plug-ins of a plurality of software plug-ins that is capable of recognizing one or more input information in the one or more images, wherein the one or more software plug-ins of the plurality of software plug-ins is capable of recognizing information contained within a document (see e.g. column 2, lines 11-32; and column 8, lines 9-61: Cooper teaches that the mobile computing device is enabled to analyze captured sensor information to identify one or more objects represented therein. Cooper particularly discloses that a recognition module performs recognition on the content captured by the mobile device to identify one or more objects therein – see e.g. column 10, line 65 – column 11, line 15; and column 17, lines 5-37. Moreover, Cooper teaches that the augmented reality system can be extended via one or more plug-ins, e.g. “matchers,” to identify particular objects within the sensor data – see e.g. column 16, lines 6-22; column 17, lines 17-37; column 20, line 60 – column 21, line 5; and column 26, lines 4-36. The plug-ins employed to recognize information can be selected automatically or by a user – see e.g. column 4, lines 34-43, column 21, lines 27-46; and column 23, line 43 – column 24, line 7. Cooper further teaches that such functionality, including the plug-ins, can be partially or entirely implemented on the mobile computing device, i.e. via the processor thereof – see e.g. column 4, lines 2-6; column 9, lines 15-24; column 10, lines 5-11; column 14, lines 12-34; column 21, lines 6-46; and column 26, lines 4-36. In such embodiments, the processor selects one or more software plug-ins of the plurality of plug-ins to recognize one or more input information, i.e. one or more objects, in the images captured by the camera of the mobile computing device. Cooper suggests that the one or more software plug-ins are particularly capable of recognizing objects contained within a document, e.g., to recognize text, phone numbers and/or web addresses within an image of a document captured by the camera – see e.g. column 1, line 62 – column 2, line 10; column 2, line 56 – column 3, line 28; column 4, lines 20-43; column 5, line 48 – column 6, line 14; column 7, lines 30-63; and FIGS. 2A-D.);
inputting, by the processor, the one or more images of the document to each of the selected one or more software plug-ins (see e.g. column 16, lines 6-22; column 17, lines 17-37; column 20, line 60 – column 21, line 5; and column 26, lines 4-36: like noted above, Cooper teaches that the augmented reality system can be extended via one or more plug-ins, e.g. “matchers,” to identify particular objects within the sensor data. To recognize the particular objects, sensor data such as images are provided to each of the plug-ins/matchers – see e.g. column 17, lines 5-37; column 21, lines 6-60; column 22, lines 4-25; and column 26, lines 4-36. As noted above, Cooper suggests that the one or more software plug-ins/matchers are capable of recognizing objects contained within a document, and so it is apparent that in such embodiments one or more images of the document are input by the processor to each of the one or more matchers/plug-ins to recognize objects therein.);
utilizing, by the processor, a first software plug-in of the one or more software plug-ins to recognize a first input information contained within the document from an image of the document, and a second software plug-in of the one or more software plug-ins to recognize a second input information contained within the document from an image of the document, wherein the first software plug-in is different from the second software plug-in and wherein the first input information is different from the second input information (see e.g. column 17, line 5 – column 18, line 5; column 21, lines 6-38; column 22, lines 4-25; and column 26, lines 4-36: Cooper teaches that the sensor data can be provided to multiple plug-ins/matchers in parallel, wherein each plug-in/matcher is configured to recognize particular objects within the sensor data, and if a matcher/plug-in recognizes an object within the sensor data, it returns a digital entity with facet data indicating the recognized object. Cooper further suggests that multiple objects can be recognized within a document from the image of the document by utilizing multiple respective plug-ins – see e.g. column 1, line 62 – column 2, line 10; column 2, line 56 – column 3, line 28; column 4, lines 20-43; column 5, line 48 – column 6, line 14; column 7, lines 30-63; and FIGS. 2A-D. In such embodiments, a first plug-in/matcher is utilized to recognize first input information, i.e. a first object, contained within the document from an image of the document, and a second plug-in/matcher is utilized to recognize a second object contained within the document from an image of the document, wherein the first plug-in/matcher is different from the second plug-in/matcher and wherein the first object recognized by the first plug-in/matcher is different from the second object recognized by the second plug-in/matcher.);
generating, by the processor, first recognized input information from the first input information, and second recognized input information from the second input information (see e.g. column 17, line 5 – column 18, line 5; column 21, lines 6-38; column 22, lines 4-25; and column 26, lines 4-36: like noted above, Cooper teaches that the sensor data can be provided to multiple plug-ins/matchers in parallel, wherein each plug-in/matcher is configured to recognize particular objects within the sensor data, and if a plug-in/matcher recognizes an object within the sensor data, it returns a digital entity with facet data indicating the recognized object. Cooper further suggests that multiple objects can be recognized within a document from the image of the document utilizing multiple respective plug-ins – see e.g. column 1, line 62 – column 2, line 10; column 2, line 56 – column 3, line 28; column 4, lines 20-43; column 5, line 48 – column 6, line 14; column 7, lines 30-63; and FIGS. 2A-D. In such embodiments where multiple objects, e.g. first and second input information, are recognized in a document by multiple respective plug-ins/matchers, a first digital entity with facet data indicating a first recognized object would be generated by a first plug-in/matcher, and a second digital entity with facet data indicating a second recognized object would be generated by a second plug-in/matcher. The first digital entity with facet data indicating the first recognized object is considered first recognized input information generated from the first input information, and the second digital entity with facet data indicating the second recognized object is considered second recognized input information generated from the second input information like claimed.);
combining, by the processor, the first recognized input information and the second recognized information obtained from the first software plug-in and the second software plug-in (As described above, Cooper teaches embodiments where multiple objects are recognized in a document by multiple respective plug-ins/matchers, wherein a first digital entity with facet data indicating a first recognized object is generated by a first plug-in/matcher, and a second digital entity with facet data indicating a second recognized object is generated by a second plug-in/matcher; the first digital entity with facet data indicating the first recognized object is considered first recognized input information, and the second digital entity with facet data indicating the second recognized object is considered second recognized input information. Cooper suggests that the digital entity and facet data are stored in memory, e.g. so that “resolvers” can identify supplemental content and/or functions corresponding to the entities – see e.g. column 17, line 34 – column 18, line 45; and column 26, lines 19-63. Accordingly, in embodiments where multiple objects are recognized in a document by multiple respective plug-ins/matchers, the entity and facet data provided by the different plug-ins/matchers would understandably be stored in memory. This can be considered combining, e.g. into the same memory, the entity and facet data provided by the different plug-ins/matchers. Cooper thus further teaches combining the first recognized input information and the second recognized input information, i.e. the first and second entity and facet data, obtained from the first plug-in/matcher and the second plug-in/matcher.);
converting, by the processor, the combined first recognized input information and second recognized information into one or more contextual messages by using the first software plug-in and the second software plug-in (As described above, Cooper teaches embodiments where multiple objects are recognized in a document by multiple respective plug-ins/matchers, wherein a first digital entity with facet data indicating a first recognized object is generated by a first plug-in/matcher, and a second digital entity with facet data indicating a second recognized object is generated by a second plug-in/matcher. Like further described above, Cooper suggests that the digital entity and facet data are stored in memory, e.g. so that “resolvers” can identify supplemental content and/or functions corresponding to the entities – see e.g. column 17, line 34 – column 18, line 45; and column 26, lines 19-63. Such supplemental content and/or functions can be considered contextual messages like claimed. Accordingly, Cooper is considered to teach converting the combined first recognized input information and second recognized input information, i.e. the first and second entity and facet data stored in memory by using the first software plug-in and the second software plug-in, into one or more contextual messages, i.e. into supplemental content and/or functions.); and
presenting, by the processor, the one or more contextual messages on a user interface displayed on the visual display (see e.g. column 2, lines 32-41: Cooper discloses that the augmented reality system operates in real-time to provide the user with the supplemental content while engaging with the primary content captured by the sensors. Like noted above, Cooper discloses that the supplemental content can comprise information and/or selectable functions associated with each object identified in the sensor information – see e.g. column 6, lines 15-57; column 8, lines 24-61; and column 18, line 46 – column 19, line 3. Cooper particularly demonstrates that the display screen of the mobile device presents the supplemental content and/or selectable functions, i.e. the one or more contextual messages corresponding to the first software plug-in/matcher and the second software plug-in/matcher, via “ribbons” overlaid on the image frames captured by the camera of the mobile device – see e.g. column 5, line 48 – column 6, line 57; column 7, lines 30-63; column 18, lines 46-66; column 26, lines 37-63; and FIGS. 2A-D.).
Accordingly, Cooper teaches a method similar to that of claim 7. Cooper discloses that such teachings can be implemented via processor-executable instructions stored on non-transitory computer-readable media, e.g. the storage of a mobile device (see e.g. column 10, lines 5-20; column 13, lines 50-60; column 26, lines 4-63; and column 28, lines 5-67). Such non-transitory computer-readable media comprising computer-executable instructions for implementing the above-described teachings of Cooper is considered a non-transitory computer-readable medium similar to that of claim 13. Cooper, however, does not explicitly disclose that the document is an identification document, and wherein the software plug-ins are capable of verifying the one or more input information contained within the identification document, as is required by claims 7 and 13.
Purves nevertheless describes a mobile computing device that (i) displays, in real-time, one or more images captured by the device, (ii) recognizes input information in the one or more images including e.g. by verifying information contained within an identification document (e.g. within a payment card, a license, a library membership card, or an insurance card), and (iii) presents one or more contextual messages (e.g. overlay labels) related to the recognized input information (see e.g. paragraphs 0205, 0209-0210, 0216, 0218-0222 and 0250-0252, and FIGS. 9A-B and 11-13).
It would have been obvious to one of ordinary skill in the art, having the teachings of Cooper and Purves before the effective filing date of the claimed invention, to modify the software plug-ins taught by Cooper, which recognize input information in a document, so as to particularly be applicable to an identification document like taught by Purves, wherein the one or more software plug-ins would be capable of recognizing and verifying information contained within the identification document. It would have been advantageous to one of ordinary skill to utilize such a combination because it would enable relevant contextual messages to be presented to the user regarding the identification document, as is demonstrated by Purves (see e.g. paragraphs 0209-0210, 0218-0222 and 0252, and FIGS. 9A-B and 11-13). Accordingly, Cooper and Purves are considered to teach, to one of ordinary skill in the art, a method like that of claim 7 and a non-transitory computer-readable medium like that of claim 13.
As per claims 8 and 14, Cooper further teaches that the one or more input information comprises at least one of a recognizable object or a symbol (see e.g. column 2, lines 11-32; and column 8, lines 9-61: like noted above, Cooper teaches that mobile computing device is enabled to analyze captured sensor information to identify one or more objects represented therein.). Purves provides a similar teaching (see e.g. paragraphs 0209-0210, 0216, 0218-0222 and 0250-0252, and FIGS. 9A-B and 11-13). Accordingly, the above-described combination of Cooper and Purves further teaches a method and non-transitory computer-readable medium like in claims 8 and 14, respectively.
As per claims 9 and 15, Cooper further teaches that the first software plug-in and the second software plug-in utilized by the processor is based on user input, and wherein the user input is indicative of user behavior (see e.g. column 23, lines 43-63: Cooper discloses that certain plug-ins can be dynamically activated based on user behavior, e.g. when the user leaves his or her home. Additionally, Cooper discloses that specific plug-ins for certain sensor data can be selected by the user – see e.g. column 21, lines 27-37. Cooper also discloses that the identification of objects within the sensor data, e.g. by the plug-ins, can be performed in response to user input indicative of user behavior, e.g. in response to gestures – see e.g. column 5, lines 35-47.). Purves similarly teaches that the identification of objects within the sensor data can be performed in response to user input indicative of user behavior (e.g. pointing gestures) (see paragraphs 0209-0210, 0216, 0218-0222 and 0250-0252). Like noted above, Cooper teaches that such recognition can be performed by one or more plug-ins (see e.g. column 16, lines 6-22; column 17, lines 17-37; column 20, line 60 – column 21, line 5; and column 26, lines 4-36). Accordingly, the above-described combination of Cooper and Purves further teaches a method and non-transitory computer-readable medium like in claims 9 and 15, respectively.
As per claims 10 and 16, Cooper further teaches that the processor is configured to utilize the first software plug-in and the second software plug-in to generate one or more information elements from the one or more input information (see e.g. column 16, lines 6-22; column 17, lines 17-37; column 20, line 60 – column 21, line 5; and column 26, lines 4-36: like noted above, Cooper teaches that one or more plug-ins, e.g. “matchers,” can be utilized to identify particular objects within the sensor data. The processor thus executes the one or more plug-ins, i.e. the first software plug-in and the second software plug-in, to generate one or more information elements from the input information, i.e. to generate indications of identified objects in the sensor data.). Accordingly, the above-described combination of Cooper and Purves further teaches a method and non-transitory computer-readable medium like in claims 10 and 16, respectively.
As per claims 11 and 17, Cooper teaches that the processor is configure to convert the one or more information elements to the one or more contextual messages (see e.g. column 11, line 16 – column 12, line 23: like noted above, Cooper discloses that a supplemental content module and a function module operate to identify supplemental content and functions related to the objects identified in the sensor content. Cooper also teaches that one or more plug-ins can be utilized to provide the supplemental content and/or functions associated with the identified objects – see e.g. column 16, lines 22-35; column 18, lines 6-45; column 19, lines 22-56; and column 26, lines 37-63. The processor thus executes the supplemental content module, the function module and/or different plug-ins to convert the one or more information elements, i.e. the identified one or more objects, to the one or more contextual messages, i.e. supplemental content and/or functions.). Accordingly, the above-described combination of Cooper and Purves further teaches a method and non-transitory computer-readable medium like in claims 11 and 17, respectively.
As per claims 12 and 18, Cooper further teaches decoding a symbol to generate the one or more information elements in an instance in which the one or more input information includes the symbol, wherein the symbol corresponds to a barcode (see e.g. column 2, lines 21-41; column 6, lines 26-39; column 17, line 65 – column 18, line 5; and column 18, lines 22-32: Cooper discloses that the objects recognized by the AR system can include bar codes, whereby the supplemental information and/or functions relate to the object. In such circumstances, the AR system understandably decodes a symbol to generate one or more information elements, wherein the symbol corresponds to a bar code.). Accordingly, the above-described combination of Cooper and Purves further teaches a method and non-transitory computer-readable medium like in claims 12 and 18, respectively.
As per claim 19, it would have been obvious, as is described above, to modify the software plug-ins taught by Cooper, which recognize input information in a document, so as to particularly be applicable to an identification document like taught by Purves. Purves particularly teaches that the identification document can be a driver’s license (see e.g. paragraphs 0219 and 0252, and FIG. 11). Accordingly, the above-described combination of Cooper and Purves further teaches a mobile computing device like that of claim 19.
Response to Arguments
The Examiner acknowledges the Applicant’s amendments to claims 1-4, 6-10 and 12-18. In response to these amendments, the 35 U.S.C. §§ 112 and 101 rejections presented in the previous Office Action regarding claims 13-18 are respectfully withdrawn. The Examiner respectfully notes, however, that the Applicant’s amendments have resulted in the new claim objections presented above.
Regarding the 35 U.S.C. § 103 rejections, the Applicant argues that Cooper and Purves fail to teach or suggest “utilize a first software plugin of the one or more software plug-ins to recognize a first input information contained within the identification document from an image of the identification document,” “utilize a second software plugin of the one or more software plug-ins to recognize a second input information contained within the identification document from an image of the identification document, wherein the first software plug-in is different from the second software plugin and wherein the first input information is different from the second input information, “generate first recognized input information from the first input information,” and “generate second recognized input information from the second input information,” as is now recited in independent claim 1 and recited similarly in independent claims 7 and 13.
The Examiner, however, respectfully disagrees. Cooper describes an augmented reality system that can be extended via one or more plug-ins to identify particular objects, and to provide supplemental content and/or functionality based on the particular objects (see e.g. column 1, line 62 – column 2, line 9; column 4, lines 20-43; and column 16, lines 6-22). Cooper suggests that the one or more software plug-ins are particularly capable of recognizing objects contained within a document, e.g., recognizing text, phone numbers and/or web addresses within an image of a document captured by the camera (see e.g. column 1, line 62 – column 2, line 10; column 2, line 56 – column 3, line 28; column 4, lines 20-43; column 5, line 48 – column 6, line 14; column 7, lines 30-63; and FIGS. 2A-D). In particular, “matcher” plug-ins are utilized to identify particular types of objects (see e.g. column 16, lines 6-22; column 17, lines 17-37; column 20, line 60 – column 21, line 5; and column 26, lines 4-36).
Cooper particularly discloses that sensor data (e.g. an image of a document) can be provided to multiple matchers in parallel to identify multiple unrelated objects; if a matcher recognizes an object within the sensor data, it returns a digital entity with facet data indicating the recognized object (see e.g. column 17, line 5 – column 18, line 5; column 21, lines 6-38; column 22, lines 4-25; and column 26, lines 4-36). As such, Cooper suggests that multiple heterogeneous objects can be recognized within a document from the image of the document utilizing multiple respective matchers (i.e. plug-ins) in parallel (see e.g. column 1, line 62 – column 2, line 10; column 2, line 56 – column 3, line 28; column 4, lines 20-43; column 5, line 48 – column 6, line 14; column 7, lines 30-63; and FIGS. 2A-D). In such embodiments, a first matcher is utilized to recognize first input information, i.e. a first object, contained within the document from an image of the document, and a second matcher is utilized to recognize a second object contained within the document from an image of the document, wherein the first matcher is different from the second matcher and wherein the first object recognized by the first matcher is different from the second object recognized by the second matcher.
Like noted above, Cooper teaches that if each matcher recognizes an object within the sensor data, it returns a digital entity with facet data indicating the recognized object (see e.g. column 17, line 5 – column 18, line 5; column 21, lines 6-38; column 22, lines 4-25; and column 26, lines 4-36). Accordingly, in the embodiments taught by Cooper where multiple objects, e.g. first and second input information, are recognized in a document by multiple respective matchers, a first matcher recognizing a particular type of object would generate a first digital entity with facet data indicating the first recognized object, and a second matcher recognizing a second particular type of object would generate a second digital entity with facet data indicating the second recognized object. The first digital entity with facet data indicating the first recognized object is considered first recognized input information generated from the first input information, and the second digital entity with facet data indicating the second recognized object is considered second recognized input information generated from the second input information like claimed.
Consequently, Cooper is considered to teach, inter alia: (i) utilizing a first software plugin (i.e. a first matcher) of one or more software plug-ins to recognize first input information (i.e. a first object) contained within a document from an image of the document; (ii) utilizing a second software plugin (i.e. a second matcher) of the one or more software plug-ins to recognize second input information (i.e. a second object) contained within the document from an image of the document, wherein the first software plug-in is different from the second software plugin and wherein the first input information is different from the second input information; (iii) generating first recognized input information (i.e. a first digital entity with facet data indicating the first recognized object) from the first input information; and (iv) and generating second recognized input information (i.e. a second digital entity with facet data indicating the second recognized object) from the second input information. However, like noted in the above rejections, Cooper does not explicitly disclose that the document is an identification document like claimed.
Nevertheless, like described in the 35 U.S.C. § 103 rejections presented above, Purves describes a mobile computing device that (i) displays, in real-time, one or more images captured by the device, (ii) recognizes input information in the one or more images including e.g. by verifying information contained within an identification document (e.g. within a payment card, a license, a library membership card, or an insurance card), and (iii) presents one or more contextual messages (e.g. overlay labels) related to the recognized input information (see e.g. paragraphs 0205, 0209-0210, 0216, 0218-0222 and 0250-0252, and FIGS. 9A-B and 11-13).
As described above, it would have been obvious to one of ordinary skill in the art, having the teachings of Cooper and Purves before the effective filing date of the claimed invention, to modify the software plug-ins taught by Cooper, which recognize input information in a document, so as to particularly be applicable to an identification document like taught by Purves, wherein the one or more software plug-ins would be capable of recognizing and verifying information contained within the identification document. It would have been advantageous to one of ordinary skill to utilize such a combination because it would enable relevant contextual messages to be presented to the user regarding the identification document, as is demonstrated by Purves (see e.g. paragraphs 0209-0210, 0218-0222 and 0252, and FIGS. 9A-B and 11-13). Accordingly, the Examiner respectfully maintains that the combination of Cooper and Purves teaches, inter alia, “utilize a first software plugin of the one or more software plug-ins to recognize a first input information contained within the identification document from an image of the identification document,” “utilize a second software plugin of the one or more software plug-ins to recognize a second input information contained within the identification document from an image of the identification document, wherein the first software plug-in is different from the second software plugin and wherein the first input information is different from the second input information, “generate first recognized input information from the first input information,” and “generate second recognized input information from the second input information,” as is recited in independent claim 1 and recited similarly in independent claims 7 and 13.
The Applicant's arguments concerning the 35 U.S.C. § 103 rejections have thus been fully considered, but are not persuasive.
Conclusion
The prior art made of record on form PTO-892 and not relied upon is considered pertinent to applicant’s disclosure. The applicant is required under 37 C.F.R. §1.111(C) to consider these references fully when responding to this action. In particular, the U.S. Patent Application Publication to Bathiche et al. cited therein generally teaches combining (i.e. consolidating and aggregating) various information in order to provide in an augmented reality experience.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BLAINE T BASOM whose telephone number is (571)272-4044. The examiner can normally be reached Monday-Friday, 9:00 am - 5:30 pm, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matt Ell can be reached at (571)270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BTB/
2/19/2026
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141