DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the Amendment filed on 6/8/2026.
Claims 1-20 are pending. Claims 1, 4, 7, 11, 14, 17 has been amended.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 8, 10-14, 18, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rodriguez et al. (US 20120224743 A1, hereinafter Rodriguez) in view of Du et al. (US 20180012102 A1, hereinafter Du), further in view of Boncyk et al. (US 7477780 B2, hereinafter Boncyk).
Regarding Claim 1, Rodriguez teaches a method of presenting poster information to a user comprising (Rodriguez, Paragraph [0385], "Consider a venue that includes a poster."; Paragraph [0384], "Auxiliary information particularly corresponding to such excerpt is then presented to the user"), enabling capture by a portable camera of an image of a poster object having indicia disposed on a visible surface area thereof (Rodriguez, Paragraph [0384], "a user can image different areas of the poster with the smartphone camera. The phone identifies salient points in the captured imagery"), the poster object visible surface area comprising a first area of recognition (Rodriguez, Paragraph [0384], "such a user can be presented one response if viewing a first part of the poster") and a second area of recognition (Rodriguez, Paragraph [0384], "a different response if viewing a second part of the poster"), [[ each comprising a respective discrete region of indicia printed on the visible surface area ]], [[ the first area of recognition and the second area of recognition each being smaller than the visible surface area ]]; based on a captured image of the poster object, recognizing the poster object from patterns imprinted in the first area of recognition and the second area of recognition (Rodriguez, Paragraph [0283], "When a smartphone camera detects reference pattern 210, it can thereby discern the relative distance between the camera and the printed object"; Paragraph [0384], "The phone identifies salient points in the captured imagery, and matches them with salient points returned from the database"), [[ without requiring the captured image to contain a hidden signature, hidden pattern, bar code or QR code ]]; matching the recognized poster object with a data record in a remote database in response to the recognizing to identify the poster object (Rodriguez, Paragraph [0383], "This poster-identifying information is transmitted by the phone to a database, which returns salient points associated with the poster."; Paragraph [0388], "in response to receipt of a watermark, NFC, or other object identifier from a smartphone, a remote server/database can return a set of SIFT or SURF data corresponding to that object."; Paragraph [0141], "When matching signs are found in the database, textual meanings associated with the discerned signs are retrieved from the database records"); selecting an interactive media offering item in response to the matching (Rodriguez, Paragraph [0101], "certain of the foregoing embodiments ease the user's dilemma of locating apps associated with certain media content. Instead, the media content serves to locate its own favored apps." <read on selecting a media offering item in response to the matching>); and superimposing the selected interactive media offering item onto a display of the captured image or an image derived therefrom (Rodriguez, Paragraph [0299], "is rendered as an on-screen graphic overlay that is triggered by smartphone detection of a signal (e.g., the FIG. 11 or 14 signal) encoded in the medium.").
But Rodriguez does not explicitly disclose each comprising a respective discrete region of indicia printed on the visible surface area, the first area of recognition and the second area of recognition each being smaller than the visible surface area.
However, Du teaches each comprising a respective discrete region of indicia printed on the visible surface area (Du, Paragraph [0032], "in some embodiments it may be possible to determine or extract multiple swatches from a determined apparel region"; "It is also possible that a single item might have regions with different patterns" <read on a respective discrete region of indicia printed on the visible surface area>; Paragraph [0020], "locating a reasonably-sized image patch, or 'swatch' containing values for a contiguous subset of pixels, from the interior of the object of interest"),
the first area of recognition and the second area of recognition each being smaller than the visible surface area (Du, Paragraph [0029], "In some embodiments the swatch is the largest region (square, rectangular, or otherwise) that will fit in a region having at least a minimum probability (e.g., at least 70% or more) of corresponding to the object of interest." <read on each being smaller than the visible surface area>; Paragraph [0020], "locating a reasonably-sized image patch, or 'swatch' containing values for a contiguous subset of pixels, from the interior of the object of interest").
Du and Rodriguez are analogous since both of them are dealing with extracting multiple discrete pattern-bearing regions of a printed-pattern object from a captured image and using those regions to match against a remote content database. Rodriguez provided a way of capturing imagery of different areas of a poster and using salient-point pattern information for matching to a remote database. Du provided a way of extracting multiple discrete swatch sub-regions each containing a distinct printed pattern and each smaller than the overall image. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the multiple discrete swatch-region extraction taught by Du into modified invention of Rodriguez such that the poster's visible surface area is treated as comprising a first and a second discrete printed region each smaller than the visible surface area and each used for pattern matching. The motivation is to improve matching robustness across differently patterned regions of the poster and to enable position-dependent responses, discussed by Rodriguez in Paragraph [0383].
But the combination of Rodriguez and Du does not explicitly disclose without requiring the captured image to contain a hidden signature, hidden pattern, bar code or QR code.
However, Boncyk teaches without requiring the captured image to contain a hidden signature, hidden pattern, bar code or QR code (Boncyk, Column 1, lines 22-28, "The detection, identification, determination of position and orientation, and subsequent information provision and communication must occur without modification or disfigurement of the object, without the need for any marks, symbols, codes, barcodes, or characters on the object").
Boncyk and Rodriguez are analogous since both of them are dealing with processing an object captured in an image and identifying that object for responsive information delivery. Rodriguez provided a way of identifying a printed object in the captured image and overlaying content on the image. Boncyk provided a way of automatically recognizing the object from within the captured image without using any additional markers, symbols, codes, or barcodes on the object. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate the markerless recognition taught by Boncyk into modified invention of Rodriguez such that the poster is recognized from its ordinarily printed indicia in the first and second areas of recognition without requiring hidden signatures, hidden patterns, bar codes, or QR codes. The motivation is to broaden the class of posters recognizable by the system and to avoid the design encumbrance of visible or hidden codes on the poster, discussed by Boncyk in Column 1, lines 22-28.
Regarding Claim 2, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination further teaches wherein the superimposing comprises using at least one of augmented reality, mixed reality and virtual reality (Rodriguez, Paragraph [0114], [0608], “Augmented reality techniques are known for recognizing image features, and overlaying information such as labels” “A central character in a virtual reality gaming world frequented by the user”).
Regarding Claim 3, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination further teaches wherein the recognition comprises recognizing a two-dimensional or three-dimensional poster object with print on it (Rodriguez, Paragraph [0391], “a user may capture imagery from a poster, decode a watermark <read on print> payload” [0123], “identified physical objects by analysis of captured image data” [0615], “employ a 3D model to generate the background image--computing appropriate 2D views based on the phone's viewpoint”).
Regarding Claim 4, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination further teaches wherein the poster object has indicia printed thereon, and the recognizing comprises recognizing at least some indicia printed in the first area of recognition (Rodriguez, Paragraph [0455], "objects may be recognized in imagery (e.g., by watermarking or fingerprinting), and the smartphone may present tags (aka icons or baubles) in association with such displayed objects"; Paragraph [0384], "The phone identifies salient points in the captured imagery"), and recognizing at least some indicia printed in the second area of recognition (Rodriguez, Paragraph [0152], “each entry includes a graphical indicia indicating the type of feature that was recognized”).
But Rodriguez does not explicitly disclose the recognizing at least some indicia printed in the second area of recognition as a discrete second-area recognition step.
However, Du teaches the recognizing at least some indicia printed in the second area of recognition (Du, Paragraph [0032], "in some embodiments it may be possible to determine or extract multiple swatches from a determined apparel region"; "Capturing multiple swatches enables multiple searches to be performed concurrently (or sequentially, etc.) in order to obtain results that match various items represented in the image.").
As explained in rejection of claim 1, the obviousness for combining of first and second region of image of Du into Rodgiguez is provided above.
Regarding Claim 8, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination further teaches wherein the superimposing is performed on a handheld display device [[, a user's retina or smart glasses ]] (Rodriguez, Paragraph [0288], “the smartphone is imaging part of a cereal box--the artwork 222 of which occupies most of the screen. Superimposed on the screen is a half-plane depiction of the detected reference signal”).
Regarding Claim 10, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination further teaches including displaying any or all of the following action buttons in any combination or subcombination (Rodriguez, Paragraph [0155], “A button control 130 on the application UI toggles such functionality on and off.”): [[ Price Tag, Photo Gallery, Videos, Description, Call, Mail, Shop link, Explanation, Intro, ]] Social Media links (Rodriguez, Paragraph [0117], “Social network theorists will recognize that this is a form of social network analysis, but with nodes representing physical objects.”), [[ Map, Discount Codes, Reviews, Tutorials, Directions, Test drives, and/or Booking opportunities ]].
Regarding Claim 11, it recites limitations similar in scope to the limitations of claim 1, but in a system. As shown in the rejection, the combination of Rodriguez, Du and Boncyk teaches the limitations in Claim 1. Additionally, Rodriguez discloses an apparatus that maps to Fig. 1 and Paragraph [0059]-[0060],(Rodriguez, Paragraph [0059]-[0060], Referring to FIG. 1, an illustrative system 12 includes a device 14 having a processor 16, a memory 18, one or more input peripherals 20, and one or more output peripherals 22. System 12 may also include a network connection 24, and one or more remote computers 26…An illustrative device 14 is a smartphone or a tablet computer, although any other consumer electronic device can be used). Thus, Claim 11 is met by Rodriguez according to the mapping presented in the rejection of claims 1, given the method corresponds to the apparatus.
Regarding Claim 12, it recites limitations similar in scope to the limitations of Claim 2 and therefore is rejected under the same rationale.
Regarding Claim 13, it recites limitations similar in scope to the limitations of Claim 3 and therefore is rejected under the same rationale.
Regarding Claim 14, it recites limitations similar in scope to the limitations of Claim 4 and therefore is rejected under the same rationale.
Regarding Claim 18, it recites limitations similar in scope to the limitations of Claim 8 and therefore is rejected under the same rationale.
Regarding Claim 20, it recites limitations similar in scope to the limitations of Claim 10 and therefore is rejected under the same rationale.
Claim(s) 5-7, 9, 15-17, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rodriguez et al. (US 20120224743 A1, hereinafter Rodriguez) in view of Du et al. (US 20180012102 A1, hereinafter Du), further in view of Boncyk et al. (US 7477780 B2, hereinafter Boncyk) as applied to Claim 1, 11 above respectively and further in view of DiBernardo et al. (US 20160100215 A1, hereinafter DiBernardo).
Regarding Claim 5, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 4.
The combination does not explicitly disclose but DiBernardo teaches wherein the recognizing includes recognizing characters printed on the object (DiBernardo, Paragraph [0019], “the mapping engine 132 can identify a specific graphic (e.g. an advertiser's logo) and/or a specific line of text”).
DiBernardo and Rodriguez are analogous since both of them are dealing with processing image data in augmented/virtual reality environment. Rodriguez provided a way of identifying object in the captured image and overlaying graphics item onto the image based on identified object in the augmented reality environment. DiBernardo provided a way of overlaying media content onto the current image based on recognition in the virtual environment by recognizing the text and graphic logo.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate text recognition taught by DiBernardo into modified invention off Rodriguez such that during the processing of image data in the augmented reality environment, system will be able to dynamically inserting useful information by recognizing different type of information including text and graphics which enhance the functionality and to provide more flexibility to the system.
Regarding Claim 6, the combination of Rodriguez, Du and Boncyk and DiBernardo teaches the invention in Claim 5.
The combination further teaches wherein the poster object comprises a printed advertisement (DiBernardo, Paragraph [0017], “objects that can appear in overlays on different advertisements, in different advertising campaigns, and/or by different advertisers”).
DiBernardo and Rodriguez are analogous since both of them are dealing with processing image data in augmented/virtual reality environment. Rodriguez provided a way of identifying object in the captured image and overlaying graphics item onto the image based on identified object in the augmented reality environment. DiBernardo provided a way of overlaying media content onto the current image based on different advertisements. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate different advertisements taught by DiBernardo into modified invention off Rodriguez such that during the processing of image data in the augmented reality environment, system will be able to dynamically inserting different kind of information including different advertisements which allow system to process more variety kind of data and enhance the enhance the functionality of the system.
Regarding Claim 7, the combination of Rodriguez, Du and Boncyk and DiBernardo teaches the invention in Claim 6.
The combination further teaches wherein the selected media item comprises a digital overlay that leads to specific action selected from the group consisting of providing specific information; a video, tutorial, or any kind of displayable content (Rodriguez, Paragraph [0150], “"whoosh" sound is then emitted from the device speaker, and an animated indicia moves from the bracketed part of the screen to a History 122 button at the bottom”; [0021], “Many apps concern media content. Some are designed to provide on-demand playback of audio or video content”).
Regarding Claim 9, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination does not explicitly disclose but DiBernardo teaches wherein the selected interactive media item comprises a call button (DiBernardo, Paragraph [0019], [0023], “Overlay 210, displayed over the images 202, has a background 211 and a button 212 retrieved from storage 103 (in accordance with the first trigger key portion 121).” “the mapping engine 132 can identify a specific graphic (e.g. an advertiser's logo) and/or a specific line of text to be inserted in the overlay with the graphic ( e.g. a "call to action" or CTA <read on call button>”).
DiBernardo and Rodriguez are analogous since both of them are dealing with processing image data in augmented/virtual reality environment. Rodriguez provided a way of identifying object in the captured image and overlaying graphics item onto the image based on identified object in the augmented reality environment. DiBernardo provided a way of overlaying media content like call button onto the current image based on recognition in the virtual environment. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate call button overlay taught by DiBernardo into modified invention off Rodriguez such that during the processing of image data in the augmented reality environment, system will be able to dynamically inserting call button on the screen based on the recognition of object/text from the image which provide user friendly environment to allow user to use more convenient way to communicate with the shops they are interested.
Regarding Claim 15, it recites limitations similar in scope to the limitations of Claim 5 and therefore is rejected under the same rationale.
Regarding Claim 16, it recites limitations similar in scope to the limitations of Claim 6 and therefore is rejected under the same rationale.
Regarding Claim 17, it recites limitations similar in scope to the limitations of Claim 7 and therefore is rejected under the same rationale.
Regarding Claim 19, it recites limitations similar in scope to the limitations of Claim 9 and therefore is rejected under the same rationale.
Response to Arguments
Applicant’s arguments with respect to claim 1,11 filed on 6/8/2026, with respect to rejection under 35 USC § 103 have been considered but are not persuasive.
Applicant asserts that Du Is Not Analogous Art to the Claimed Invention.
In response to the argument, The analogous-arts test is disjunctive, a reference is analogous if it is either in the same field or reasonably pertinent to the problem faced by the inventor. The Examiner relies on the "reasonably pertinent" prong. The claimed invention's problem is: given a captured image of a printed object bearing indicia over a visible surface area, extract multiple discrete regions of the printed indicia and match those regions to a database record to identify the object. That is the very problem Du addresses at a level of generality that a POSITA would recognize, i.e. extracting multiple discrete pattern-bearing sub-regions ("swatches") from a captured image of a printed-pattern object for database matching (Du [0020], [0029], [0032]). Du's algorithms for locating body pose and torso are Du's particular application; the technique of extracting multiple discrete printed-pattern regions from a captured object image is the pertinent teaching relied upon. A person having ordinary skill in the art seeking to improve match robustness in Rodriguez's already-established poster-recognition pipeline (Rodriguez [0384]) would reasonably consult references teaching multi-region pattern extraction, of which Du is one. Therefore, applicant remark cannot be considered persuasive.
Applicant further asserts that Du's "Swatches" Are Not "Areas of Recognition" As Claimed.
In response to the argument, The claim does not require that the poster designer physically delineate the areas of recognition, the claim requires that the visible surface area comprise first and second areas of recognition, each comprising a discrete region of indicia printed on the visible surface. Under the broadest reasonable interpretation, an area is a discrete region of the printed surface from which the system extracts indicia patterns for recognition; it is not required to be pre-marked or bounded by a physical border. Rodriguez [0384] already teaches imaging different areas of the poster and extracting salient points from each, i.e. the areas are defined by what is imaged/processed. Du's swatches likewise are discrete regions of the printed indicia on the object's visible surface used for matching. The claim as amended does not distinguish over this reading. Furthermore, the claim recites "matching the recognized poster object with a data record in a remote database…to identify the poster object." Rodriguez expressly teaches identifying the specific poster: "This poster identifying information is transmitted by the phone to a database, which returns salient points associated with the poster" (Rodriguez [0383]); "in response to receipt…of an object identifier from a smartphone, a remote server/database can return a set of SIFT or SURF data corresponding to that object" (Rodriguez [0388]). The identification-of-specific-object limitation is met by Rodriguez, not by Du. Du is relied upon only for the multi-discrete-region extraction feature. The applicant's argument that identification is categorically different from similarity matching is therefore misplaced — the identification comes from Rodriguez; Du contributes only the multi-region matching primitive. Therefore, applicant remark cannot be considered persuasive.
Applicant further asserts that No Valid Motivation to Combine Rodriguez and Du.
In response to the argument, The motivation articulated is not that Rodriguez's recognition is deficient in an absolute sense, but that multi-region matching provides robustness benefits and enables position-dependent interaction, a benefit Rodriguez itself invites ([0383]: "The user can then interact with the poster in a position dependent manner"). Du need not be bodily transplanted into Rodriguez; only the pertinent teaching, i.e. extracting multiple discrete printed-pattern regions from a captured object image and using each as a matching input and need be combined. Body-pose detection is Du's application-specific preprocessing and is neither claimed nor needed for Rodriguez's poster context. The combination is at the level of the multi-region-extraction technique, not the full Du pipeline i.e. the combination need not be a "bodily incorporation"; obviousness may rest on the applied teaching). Therefore, applicant remark cannot be considered persuasive.
Applicant further asserts that the Rodriguez + Du + Boncyk Triple Combination Has Compounding Flaw.
In response to applicant's argument that the examiner's conclusion of obviousness, with respect to rejection under 35 USC § 103 is based upon improper hindsight reasoning. As a matter of fact, The Examiner's articulated motivation for each combination step is (i) grounded in the references' own teachings — Rodriguez itself invites position-dependent interaction with the poster ([0383]), and Boncyk itself articulates the desirability of markerless recognition to avoid disfiguring the object (Col. 1, ll. 22-28) — and (ii) tied to advantages recognizable at the time of invention (matching robustness; broader class of recognizable posters; no design encumbrance from visible codes). It must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). Therefore, applicant remark cannot be considered persuasive.
Applicant further asserts that The Dependent Claim Rejections Fall With the Independent Claims - and Have Additional Flaws.
In response to the argument, regarding Claims 10/20: The claim recites "displaying any or all of the following action buttons in any combination or subcombination", the claim is satisfied by displaying any one of the listed items. Rodriguez [0117] teaches social-network / social-media related overlays with nodes representing physical objects; Rodriguez [0155] teaches button controls on the application UI. Together with DiBernardo's CTA/call-button teaching ([0019], [0023]), the "any or all…any combination or subcombination" language is met by even a single button. Regarding Claims 7/17: Rodriguez [0021] ("Many apps concern media content. Some are designed to provide on-demand playback of audio or video content"), [0150] (media-selected overlay triggered by recognition), teach a digital overlay leading to a video/tutorial/displayable content action. Regarding Claims 5/15: DiBernardo [0019] teaches recognition of "a specific line of text" which reads on "recognizing characters printed on the object" under BRI. The addition of DiBernardo does not "further weaken" motivation; DiBernardo, like Rodriguez, is in the AR/overlay-on-captured-image field, and the motivation (enrich recognition modalities to cover both graphic-pattern and character-based indicia) is well-supported. Because none of Applicant's arguments identifies a limitation not met by the cited combination, and because the amendments "each comprising a respective discrete region of indicia printed on the visible surface area" and "to identify the poster object" do not distinguish over the combined teachings of Rodriguez ([0383]–[0388]) and Du ([0020], [0029], [0032]), the rejection of claims 1-20 is maintained on the modified grounds set forth above. Therefore, applicant remark cannot be considered persuasive.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 8638363 B2 Automatically capturing information, such as capturing information using a document-aware device.
US 9542687 B2 Systems and methods for visual representation of offers
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUJANG TSWEI whose telephone number is (571)272-6669. The examiner can normally be reached 8:30am-5:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YuJang Tswei/Primary Examiner, Art Unit 2614
Prosecution Insights