DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the Amendment filed on 11/04/2025.
Claims 1-20 are pending. Claims 1, 4, 7, 11, 14, 17 has been amended.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 11/04/2025 has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 8, 10-14, 18, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rodriguez et al. (US 20120224743 A1, hereinafter Rodriguez) in view of Du et al. (US 20180012102 A1, hereinafter Du), further in view of Boncyk et al. (US 7477780 B2, hereinafter Boncyk).
Regarding Claim 1, Rodriguez teaches a method of presenting poster information to a user (Rodriguez, Paragraph [0385], “determination methods much finer in resolution than…a venue that includes a poster”) comprising:
enabling capture by a portable camera of an image of a poster object having indicia disposed (Rodriguez, Paragraph [0384], “a user can image different areas of the poster with the smartphone camera. The phone identifies salient points in the captured imagery”; [0152], “display of previously-captured features is presented” “each entry includes a graphical indicia indicating the type of feature that was recognized”), on a visible surface area thereof, the poster object defining [[ first and second ]] areas of recognition within the visible surface area, [[ the first and second areas of recognition each being smaller than the visible surface area ]] (Rodriguez, Paragraph [0383], “This poster identifying information is transmitted by the phone to a database, which returns salient points associated with the poster”; “The user can then interact with the poster in a position dependent
manner”); based on a captured image of the poster object, recognizing the poster object from patterns imprinted in the first area of recognition [[ and the second area of recognition ]] image [[ without requiring the captured image to contain a hidden signature, hidden pattern, bar code or QR code ]] (Rodriguez, Paragraph [0283], “When a smartphone camera detects reference pattern 210, it can thereby discern the relative distance between the camera and the printed object” [0302], “The flag layer is an indicia (typically transitory) that is presented to the user as a consequence of some initial digital image processing”); matching the recognized poster object with a data record in a remote database in response to the recognizing (Rodriguez, Paragraph [0136], [0152], “a graphical indicia indicating the type of feature that was recognized, together with information discerned from the feature, and the time the feature was detected” “The indicated display features can naturally be augmented by other graphical indicia and controls associated with the smartphone functionality being used”; [0141], “When matching signs are found in the database, textual meanings associated with the discerned signs are retrieved from the database records and can be output” [0383], “This poster identifying information is transmitted by the phone to a database, which returns salient points associated with the poster. The user can then interact with the poster in a position dependent manner”; [0388], That is, in response to receipt…other object identifier from a smartphone, a remote server/database can return a set of SIFT or SURF data corresponding to that object”); selecting an [[ interactive ]] media item in response to the matching (Rodriguez, Paragraph [0101], “it will be recognized that certain of the foregoing embodiments ease the user's dilemma of locating apps associated with certain media content”); and superimposing the selected [[ interactive media ]] item onto a display of the captured image or an image derived therefrom (Rodriguez, Paragraph [0299], “the original physical medium, but is rendered as an on-screen graphic overlay that is triggered by smartphone detection of a signal” “encoded in the medium”).
But Rodriguez does not explicitly disclose on a visible surface area thereof, [[ the poster object defining ]] first and second [[ areas of recognition within the visible surface area ]] , the first and second areas of recognition each being smaller than the visible surface area [[ superimposing the selected ]] interactive media [[ item onto a display of the captured image or an image derived therefrom ]].
However, Du teaches a visible surface area thereof (Du, Paragraph [0018], "an image 102 is obtained that includes a representation of a woman wearing a dress") the poster object defining a first area of recognition and a second area of recognition (Du, Paragraph [0032], "it may be possible to determine or extract multiple swatches from a determined apparel region. <read on first area and second area>") each being smaller than the visible surface area (Du, Paragraph [0029], "the match is performed against the swatch region instead of against the entire received image" [0020], "locating a reasonably-sized image patch, or "swatch" containing values for a contiguous subset of pixels, from the interior of the object of interest").
Du and Rodriguez are analogous since both of them are dealing with
recognizing/locating regions of interest in captured imagery for matching to remote
content. Rodriguez provided a way of capturing imagery of a poster and using
pattern/salient point information for matching to a remote database. Du provided a way
of determining multiple distinct swatch regions (e.g., regions having different patterns)
within an object region and using those multiple regions for matching/search. Therefore,
it would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention was made to incorporate multiple swatch-region (i.e.,
first and second area) identification/selection taught by Du into modified invention of
Rodriguez such that the poster object defines multiple areas of recognition (each
smaller than the overall visible surface) from which patterns are used for matching ,
thereby improving matching robustness when different portions of the poster include different patterns/printed content and improving user experience by enabling
selection/use of the precise area(s) of interest.
But the combination does not explicitly disclose in the area of recognition image without requiring the captured image to contain a hidden signature, hidden pattern, bar code or QR code
But, Boncyk teaches based on the captured image, superimpose the area of recognition image (Boncyk, Column 2, Line 15-17, capturing imagery of the objects and then identifying the objects via image recognition performed on a local or remote computer) without requiring the captured image area of recognition to contain a hidden signature, hidden pattern, bar code or QR code (Boncyk, Column 1, Line 22-28, “The detection, identification, determination of position and orientation, and subsequent information provision and communication must occur without modification or disfigurement of the object, without the need for any marks, symbols, codes, barcodes, or characters on the object”). selecting an interactive media item in response to the matching (Boncyk, Column 23, Line 10- 14, “interactive television programming in which users "point and click" on the television screen at specific times, based on the on-screen content, to register votes, indicate actions, or connect to a web site”
Boncyk and Rodriguez are analogous since both of them are dealing with processing object captured in the image. Rodriguez provided a way of identifying object in the captured image and overlaying graphics item onto the image based on identified object in the augmented reality environment. Boncyk provided a way to automatically recognizing the object from within the image captured without using any additional helper like markers, barcode…etc. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate automatic object recognition taught by Boncyk into modified invention off Rodriguez such that during the processing of image data in the augmented reality environment, system will be able to automatically identify the object of interest from the captured image which enhance the system functionality and provided with more user friendly system.
Regarding Claim 2, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination further teaches wherein the superimposing comprises using at least one of augmented reality, mixed reality and virtual reality (Rodriguez, Paragraph [0114], [0608], “Augmented reality techniques are known for recognizing image features, and overlaying information such as labels” “A central character in a virtual reality gaming world frequented by the user”).
Regarding Claim 3, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination further teaches wherein the recognition comprises recognizing a two-dimensional or three-dimensional poster object with print on it (Rodriguez, Paragraph [0391], “a user may capture imagery from a poster, decode a watermark <read on print> payload” [0123], “identified physical objects by analysis of captured image data” [0615], “employ a 3D model to generate the background image--computing appropriate 2D views based on the phone's viewpoint”).
Regarding Claim 4, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination further teaches wherein the poster object has indicia printed thereon, and the recognizing comprises recognizing at least some indicia printed in the first area of recognition and then recognizing at least some indicia printed in the second area of recognition (Rodriguez, Paragraph [0455], “it will be recognized that objects may be recognized in imagery (e.g., by watermarking or fingerprinting), and the smartphone may present tags (aka icons or baubles) in association with such displayed objects” [0152], “each entry includes a graphical indicia indicating the type of feature that was recognized”).
Rodgiguez does not explicitly disclose but Du teaches first area of recognition and … the second area of recognition (Du, Paragraph [0032], "it may be possible to determine or extract multiple swatches from a determined apparel region. <read on first area and second area>") each being smaller than the visible surface area (Du, Paragraph [0029], "the match is performed against the swatch region instead of against the entire received image" [0020], "locating a reasonably-sized image patch, or "swatch" containing values for a contiguous subset of pixels, from the interior of the object of interest").
As explained in rejection of claim 1, the obviousness for combining of first and second region of image of Du into Rodgiguez is provided above.
Regarding Claim 8, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination further teaches wherein the superimposing is performed on a handheld display device [[, a user's retina or smart glasses ]] (Rodriguez, Paragraph [0288], “the smartphone is imaging part of a cereal box--the artwork 222 of which occupies most of the screen. Superimposed on the screen is a half-plane depiction of the detected reference signal”).
Regarding Claim 10, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination further teaches including displaying any or all of the following action buttons in any combination or subcombination (Rodriguez, Paragraph [0155], “A button control 130 on the application UI toggles such functionality on and off.”): [[ Price Tag, Photo Gallery, Videos, Description, Call, Mail, Shop link, Explanation, Intro, ]] Social Media links (Rodriguez, Paragraph [0117], “Social network theorists will recognize that this is a form of social network analysis, but with nodes representing physical objects.”), [[ Map, Discount Codes, Reviews, Tutorials, Directions, Test drives, and/or Booking opportunities ]].
Regarding Claim 11, it recites limitations similar in scope to the limitations of claim 1, but in a system. As shown in the rejection, the combination of Rodriguez, Du and Boncyk teaches the limitations in Claim 1. Additionally, Rodriguez discloses an apparatus that maps to Fig. 1 and Paragraph [0059]-[0060],(Rodriguez, Paragraph [0059]-[0060], Referring to FIG. 1, an illustrative system 12 includes a device 14 having a processor 16, a memory 18, one or more input peripherals 20, and one or more output peripherals 22. System 12 may also include a network connection 24, and one or more remote computers 26…An illustrative device 14 is a smartphone or a tablet computer, although any other consumer electronic device can be used). Thus, Claim 11 is met by Rodriguez according to the mapping presented in the rejection of claims 1, given the method corresponds to the apparatus.
Regarding Claim 12, it recites limitations similar in scope to the limitations of Claim 2 and therefore is rejected under the same rationale.
Regarding Claim 13, it recites limitations similar in scope to the limitations of Claim 3 and therefore is rejected under the same rationale.
Regarding Claim 14, it recites limitations similar in scope to the limitations of Claim 4 and therefore is rejected under the same rationale.
Regarding Claim 18, it recites limitations similar in scope to the limitations of Claim 8 and therefore is rejected under the same rationale.
Regarding Claim 20, it recites limitations similar in scope to the limitations of Claim 10 and therefore is rejected under the same rationale.
Claim(s) 5-7, 9, 15-17, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rodriguez et al. (US 20120224743 A1, hereinafter Rodriguez) in view of Du et al. (US 20180012102 A1, hereinafter Du), further in view of Boncyk et al. (US 7477780 B2, hereinafter Boncyk) as applied to Claim 1, 11 above respectively and further in view of DiBernardo et al. (US 20160100215 A1, hereinafter DiBernardo).
Regarding Claim 5, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 4.
The combination does not explicitly disclose but DiBernardo teaches wherein the recognizing includes recognizing characters printed on the object (DiBernardo, Paragraph [0019], “the mapping engine 132 can identify a specific graphic (e.g. an advertiser's logo) and/or a specific line of text”).
DiBernardo and Rodriguez are analogous since both of them are dealing with processing image data in augmented/virtual reality environment. Rodriguez provided a way of identifying object in the captured image and overlaying graphics item onto the image based on identified object in the augmented reality environment. DiBernardo provided a way of overlaying media content onto the current image based on recognition in the virtual environment by recognizing the text and graphic logo.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate text recognition taught by DiBernardo into modified invention off Rodriguez such that during the processing of image data in the augmented reality environment, system will be able to dynamically inserting useful information by recognizing different type of information including text and graphics which enhance the functionality and to provide more flexibility to the system.
Regarding Claim 6, the combination of Rodriguez, Du and Boncyk and DiBernardo teaches the invention in Claim 5.
The combination further teaches wherein the poster object comprises a printed advertisement (DiBernardo, Paragraph [0017], “objects that can appear in overlays on different advertisements, in different advertising campaigns, and/or by different advertisers”).
DiBernardo and Rodriguez are analogous since both of them are dealing with processing image data in augmented/virtual reality environment. Rodriguez provided a way of identifying object in the captured image and overlaying graphics item onto the image based on identified object in the augmented reality environment. DiBernardo provided a way of overlaying media content onto the current image based on different advertisements. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate different advertisements taught by DiBernardo into modified invention off Rodriguez such that during the processing of image data in the augmented reality environment, system will be able to dynamically inserting different kind of information including different advertisements which allow system to process more variety kind of data and enhance the enhance the functionality of the system.
Regarding Claim 7, the combination of Rodriguez, Du and Boncyk and DiBernardo teaches the invention in Claim 6.
The combination further teaches wherein the selected media item comprises a digital overlay that leads to specific action selected from the group consisting of providing specific information; a video, tutorial, or any kind of displayable content (Rodriguez, Paragraph [0150], “"whoosh" sound is then emitted from the device speaker, and an animated indicia moves from the bracketed part of the screen to a History 122 button at the bottom”; [0021], “Many apps concern media content. Some are designed to provide on-demand playback of audio or video content”).
Regarding Claim 9, the combination of Rodriguez, Du and Boncyk teaches the invention in Claim 1.
The combination does not explicitly disclose but DiBernardo teaches wherein the selected interactive media item comprises a call button (DiBernardo, Paragraph [0019], [0023], “Overlay 210, displayed over the images 202, has a background 211 and a button 212 retrieved from storage 103 (in accordance with the first trigger key portion 121).” “the mapping engine 132 can identify a specific graphic (e.g. an advertiser's logo) and/or a specific line of text to be inserted in the overlay with the graphic ( e.g. a "call to action" or CTA <read on call button>”).
DiBernardo and Rodriguez are analogous since both of them are dealing with processing image data in augmented/virtual reality environment. Rodriguez provided a way of identifying object in the captured image and overlaying graphics item onto the image based on identified object in the augmented reality environment. DiBernardo provided a way of overlaying media content like call button onto the current image based on recognition in the virtual environment. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to incorporate call button overlay taught by DiBernardo into modified invention off Rodriguez such that during the processing of image data in the augmented reality environment, system will be able to dynamically inserting call button on the screen based on the recognition of object/text from the image which provide user friendly environment to allow user to use more convenient way to communicate with the shops they are interested.
Regarding Claim 15, it recites limitations similar in scope to the limitations of Claim 5 and therefore is rejected under the same rationale.
Regarding Claim 16, it recites limitations similar in scope to the limitations of Claim 6 and therefore is rejected under the same rationale.
Regarding Claim 17, it recites limitations similar in scope to the limitations of Claim 7 and therefore is rejected under the same rationale.
Regarding Claim 19, it recites limitations similar in scope to the limitations of Claim 9 and therefore is rejected under the same rationale.
Response to Arguments
Applicant’s arguments with respect to claim 1,11 filed on 11/04/2025, with respect to rejection under 35 USC § 103 have been considered but are moot in view of the new ground(s) of rejection. it has now been taught by the combination of Rodriguez, Du and Boncyk.
In regard to Claims 2-10, 12-20 they directly/indirectly depends on independent Claim 1, 11 respectively. Applicant does not argue anything other than the independent claim 1, 11. The limitations in those claims in conjunction with combination previously established as explained.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YUJANG TSWEI whose telephone number is (571)272-6669. The examiner can normally be reached 8:30am-5:30pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached on (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YuJang Tswei/Primary Examiner, Art Unit 2614