DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application is being examined under the pre-AIA first to invent provisions.
Response to Arguments
Applicant’s terminal disclaimer filed and approved on February 27, 2026 have obviated the double patent rejections against the claims, and accordingly the double patenting rejections are withdrawn.
Applicant’s arguments and amendments in the Amendment filed February 27, 2026 (herein “Amendment”), with respect to the rejection(s) of pending independent claim(s) 2, 9 and 15, and claims depending therefrom under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Szeliski et al., US Patent Application Publication No. US 2006/0083405 A1.
Claim Objections
Claims 5 and 12, and therefore 6 and 13 which depend therefrom, are objected to because of the following informalities: claims 5 and 12 recite “including at least image data,” but “image data” is earlier recited in the independent claims from which claims 5 and 12 depend, and therefore, this limitation instead should be recited as “including at least the image data.” Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2–21 are rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Lee et al., US Patent Application Publication No. US 2008/0069404 A1 (herein “Lee”) in view of Szeliski et al., US Patent Application Publication No. US 2006/0083405 A1 (herein “Szeliski”).
Regarding claims 2, 9 and 15, with substantive differences between the claims noted in curly brackets {}, deficiencies of Lee noted in square brackets [], and with claim 2 as exemplary, Lee teaches {A method comprising: at an electronic device with one or more processors, one or more sensors, memory, and one or more displays: - claim 2 / A computing system comprising: one or more processors; one or more sensors; memory; one or more displays; and memory storing instructions, which when executed by the one or more processors, cause the computing system: - claim 9 / A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to: - claim 15 (Lee figs. 1 and 11, ¶¶ 36–42, and 85, a system and operations of the system which can be executed by instructions stored on a computer readable medium, of indexing an image object, the system including an image sensor as a camera in a portable device as a smart phone including a display, where a smart phone is known to have a processor and memory)}:
detecting, using the one or more sensors, data, including at least image data, corresponding to a first object of a plurality of objects in an environment (Lee ¶¶ 41–42, an object in the image (at least image data) received from the camera of the portable device is identified (detecting));
determining, using the one or more processors, a location of the electronic device in the environment (Lee ¶¶ 42, 53 and 44–46, spatial information corresponding to the image is computed and the geographical object location of the object is estimated thereto, including a longitude, latitude and altitude of a current location (thus “in the environment”), and where the angle calculations are determined by measuring a distance from the camera (part of the electronic device) to a subject object);
identifying, using one or more processors, the first object based on the data [including at least the image data] corresponding to the first object and the location of the electronic device in the environment, [wherein identifying the first object includes applying pattern recognition to identify the first object in the image data corresponding to the first object] (Lee ¶ 61, context information corresponding to the identified geographical object is extracted from a context database, and the information includes identifying data on the object including a geographical object name, real estate information, advertisement information and the like corresponding to the geographical object); and
presenting, using the one or more displays, a representation of one or more of the plurality of objects in the environment including the first object (Lee ¶¶ 62–65, fig. 8, display unit displays the context information (a representation) on a position of the image corresponding to the geographical object) using a list of the plurality of objects in the environment that is stored in the memory (Lee ¶ 79, and 61, context database storing the context information of the object, and map database storing the geographical location, range, name and map data information (list of the plurality of objects) is referred to (using) for indexing an image object and detecting the object).
Although Lee discloses the map database 330 as being part of the geographical object identification unit 120, part of the system of indexing an image object 100 (see Figs. 1 and 3), Lee does not explicitly teach that the map database 330 is comprised within the electronic device, rather, Lee generally teaches that the map database 330 is part of the overall system which includes the electronic device. Accordingly, Lee is not found to anticipate the claimed “list … stored in the memory” where the memory, as claimed, is part of the electronic device. Notwithstanding, it would have been obvious to a person having ordinary skill in the art at the time the invention of the present application was made (this being a pre-AIA case, with the first to invent standard applied for obviousness, rather than the first to invent standard), to have modified the system of Lee to include the map database 330 as part of the smart phone device of Lee at least because a smart phone typically has a type of local storage from which applications and data used by the smart phone are accessed, and therefore such a modification would be simply taking disparate components and making them integral with predictable results, and. See MPEP §2144.04(V)(B).
Further, Lee does not explicitly teach where Szeliski teaches identifying based on including at least the image data (Szeliski ¶¶43, 46, 49 and 50, image and template is acquired, and the image pixel data is compared to the template for whether a match is present, where the template represents data of an object to be detected), and wherein identifying the first object includes applying pattern recognition to identify the first object in the image data corresponding to the first object (Szeliski ¶¶46, 49 and 50, template image which represents an object is compared to input image for comparison (pattern recognition), wherever the comparison metric is maximized, the object is determined to be located in the image).
Therefore, taking the teachings of Lee and Szeliski together as a whole, it would have been obvious to a person having ordinary skill in the art at the time the invention of the present application was made to have modified the object identification of Lee to include pattern matching on the image data as taught in Szeliski at least because doing so would allow for high parallelization and thus efficiency of the image processing. See Szeliski ¶35.
Regarding claims 3, 10, and 16, with claim 3 as exemplary, Lee teaches wherein determining the location of the electronic device in the environment is based on location circuitry (Lee ¶¶ 45, 53, location information about the camera (part of the electronic device) is determined from a global positioning system (location circuitry)).
Regarding claims 4, 11 and 17, with claim 4 as exemplary, Lee teaches wherein the one or more sensors include one or more camera sensors, and wherein detecting the data, including at least the image data, corresponding to the first object comprises: capturing, using the one or more camera sensors, one or more images of the first object (Lee ¶¶ 41–42, an object present in the image received from the camera of the portable device is identified (detecting the data)).
Regarding claims 5, 12, and 18, with claim 5 as exemplary and deficiencies of Lee noted in square brackets [], Lee teaches further comprising: detecting, using the one or more sensors, data, including at least image data, corresponding to a second object of the plurality of objects in the environment (Lee ¶¶ 62–65, fig. 8, display unit displays the context information (a representation) on a position of the image (image data) corresponding to the geographical object, where fig. 8 illustrates multiple buildings (second object)); identifying, using the one or more processors, the second object based on the data, [including at least the image data] corresponding to the second object and the location of the electronic device in the environment [wherein identifying the second object includes applying the pattern recognition to identify the second object in the image data corresponding to the second object] (Lee ¶ 61, context information corresponding to the identified geographical object (including plural and thus second objects in the image) is extracted from a context database, and the information includes identifying data on the object including a geographical object name, real estate information, advertisement information and the like corresponding to the geographical object); and presenting, using the one or more displays, the representation of the one or more of the plurality of objects in the environment including the second object (Lee ¶¶ 62–65, fig. 8, display unit displays the context information (a representation) on a position of the image corresponding to the geographical object).
Claim 18 further recites in addition to the same limitations above given for claims 5 and 12, with Lee teaching these additional limitations: using the list of the plurality of objects that is stored in the memory (Lee ¶ 79, and 61, context database storing the context information of the object, and map database storing the geographical location, range, name and map data information (list of the plurality of objects) is referred to (using) for indexing an image object and detecting the object).
Lee does not explicitly teach where Szeliski teaches identifying based on including at least the image data (Szeliski ¶¶43, 46, 49 and 50, image and template is acquired, and the image pixel data is compared to the template for whether a match is present, where the template represents data of an object to be detected), and wherein identifying the second object includes applying the pattern recognition to identify the second object in the image data corresponding to the second object (Szeliski ¶¶46, 49 and 50, template image which represents an object to be tracked is compared to input image for comparison (pattern recognition), wherever the comparison metric is maximized, the object is determined to be located in the image, where ¶5 teaches plural objects in an image to be tracked, thus a second object and template respective thereto).
Therefore, taking the teachings of Lee and Szeliski together as a whole, it would have been obvious to a person having ordinary skill in the art at the time the invention of the present application was made to have modified the object identification of Lee to include pattern matching on the image data as taught in Szeliski at least because doing so would allow for high parallelization and thus efficiency of the image processing. See Szeliski ¶35.
Regarding claims 6, 13 and 19, with claim 6 as exemplary, Lee teaches further comprising: presenting, using the one or more displays, a representation of the first object in the environment and a representation of the second object in the environment, wherein the representation of the first object and the representation of the second object are displayed concurrently by the one or more displays (Lee fig. 8, ¶¶ 63–67, the multiple context information (representations) are placed in the position in the image where the objects (buildings) appear in the image, and as shown in fig. 8, the respective context information for each building (object) is displayed concurrently).
Regarding claims 7, 14 and 20, with claim 7 as exemplary, Lee teaches wherein the list of the plurality of objects in the environment that is stored in the memory corresponds to the location of the electronic device (Lee ¶¶ 79, 61, 45, the map database containing range information and context database storing the context information of the object, both correspond to the location of the electronic device, as the map database stores geographical distance location, where distance from the electronic device is calculated by a distance sensor and GPS, and where the context information includes position on the image from the electronic device camera viewpoint/image to place the context information).
Regarding claims 8, and 21, with claim 8 as exemplary, Lee teaches wherein determining the location of the electronic device includes identifying an object in the environment associated with the list of the plurality of objects in the environment that corresponds to the location of the electronic device (Lee ¶¶ 54, 79, geographical object detection involves detecting the geographical object included in the estimated object location by referring to the predetermined map database (list of the plurality of objects in the environment), where the map database corresponds to the location of the electronic device, as the map database stores geographical distance location).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Yang et al., US 2009/0002489 A1, directed towards visual object tracking in a crowded scene using image analysis.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M KOETH whose telephone number is (571)272-5908. The examiner can normally be reached Monday-Thursday, 09:00-17:00, Friday 09:00-13:00, EDT/EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MICHELLE M. KOETH
Primary Examiner
Art Unit 2671
/MICHELLE M KOETH/Primary Examiner, Art Unit 2671
Prosecution Insights