DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: Select the best reference image to compare against a current image by matching location, weather, time, and object types, or absolute or relative object sizes.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4 and 5 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claims 4 and 5 recite limitations, “at least one of types of classes, sizes of regions of the respective classes, and proportions of the regions of the respective classes with respect to the entire image”. However, claim 1 recites these limitations are pertaining to both the captured image and the comparison target image candidates. Yet, each of claims 4 and 5 do not make clear which of these images these limitations pertain to. Thus, one can not ascertain the metes and bound of the claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to mental process, including recited math for claim 12, abstract idea without significantly more.
Claim(s) 1 recite(s):
“analyze a captured image captured … and classify objects included in the captured image into a plurality of classes”, which can reasonably be interpreted as a human observer viewing a displayed image and mentally classifying objects in the captured image via visual perception;
“extract comparison target image candidates for comparison with the captured image… based on imaging information of the captured image, the information storage being configured to store one or more past images captured in the past, and imaging information including at least one of imaging date and time information and weather information at an imaging time for each of the one or more past images”, which can reasonably be interpreted as a human observer viewing a displayed image and mentally extracting reference image candidates based on either: imaging date and time OR weather or both;
“select a comparison target image from among the comparison target image candidates based on at least one of types of the classes included in the captured image, sizes of regions of the respective classes, and proportions of the regions of the respective classes with respect to the entire image, and at least one of types of classes included in each of the comparison target image candidates, sizes of regions of the respective classes, and proportions of the regions of the respective classes with respect to the entire image”, which can reasonably be interpreted as a human observer viewing a displayed image and mentally selecting reference image from among reference image candidates based on: types of classes, sizes of class regions, or proportion class regions relative to entire image – for both the captured image and reference image candidates;
This judicial exception is not integrated into a practical application because additional elements of:
“An information processing apparatus comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions” are generically recited computer elements that do not add a meaningful limitation to the abstract idea because they amount to simply implementing the abstract idea on a computer;
“captured image captured by using a camera mounted on a mobile object” are generically recited insignificant extra-solution activity of data gathering; and
“information storage being configured to store one or more past images captured in the past” are generically recited insignificant extra-solution activity of data gathering.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because additional elements of:
“An information processing apparatus comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions” are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f);
“captured image captured by using a camera mounted on a mobile object” are insignificant extra-solution activity of data gathering; and
“information storage being configured to store one or more past images captured in the past” are insignificant extra-solution activity of data gathering.
Depending claims do not remedy these deficiencies:
Claims 2, 3, 6, 7, 8, 9, 10, 11, 15, and 16 further recite limitations that can be reasonably be interpreted as a human observer viewing displayed images and performing mental processes via visual perception.
Claims 4, 5, 13, 14 further recite limitations that are additional elements that are generically recited insignificant extra-solution activity of data gathering.
Claim 12 further recites limitations that are recited math and can also be reasonably be interpreted as a human observer viewing displayed images and performing mental processes via visual perception.
As per claim(s) 17, arguments made in rejecting claim(s) 1 are analogous. Note that claim 17 is a method claim and does not require the “An information processing apparatus comprising: at least one memory storing instructions; and at least one processor configured to execute the instructions to” additional element limitations of claim 1.
As per claim(s) 18, arguments made in rejecting claim(s) 1 are analogous. Claim 18 also recites, “A non-transitory computer-readable medium storing a program for causing a computer to execute”, which are generically recited computer elements that do not add a meaningful limitation to the abstract idea because they amount to simply implementing the abstract idea on a computer and are mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-18 are rejected under 35 U.S.C. 103 as being unpatentable over a first embodiment of US 2020/0401617 A1 (Spiegel1) in view of a second embodiment of US 2020/0401617 A1 (Spiegel2).
As per claim 1, Spiegel1 teaches an information processing apparatus comprising:
at least one memory storing instructions; and at least one processor configured to execute the instructions to (Spiegel: Fig. 1; paras 117-126;
PNG
media_image1.png
451
717
media_image1.png
Greyscale
):
analyze a captured image captured by using a camera mounted on a mobile object, and classify objects included in the captured image into a plurality of classes (Spiegel1:
Fig. 1 (shown above): mainly 103;
Para 70: “It is one object of the invention to facilitate abstraction, indexing and/or recognition of images and to take advantage of characteristics frequently present in images of locations of interest.”;
“[0071] The system provides robust and enhanced abilities applicable in general circumstances including images of building and manmade structures and particularly useful in difficult to process scenarios such as images not having sufficient distinct characteristics to be easily differentiated. Repetitive straight lines, orthogonal elements, corners and other shapes which may be present in almost any environment, city, country, and nature can be differentiated with a higher level of confidence.”;
“[0083] In operation, a query video stream may be captured by a vehicle mounted camera. A significant mode of operation involves analyzing images in a query video stream to determine the closest match to a particular reference image”;
“[0072] One process for characterizing an image is to use feature extraction. The system may be connected to a database containing reference image extractions feature maps and a neural network specifically trained to identify correspondence between a query input and a reference entry in a database. The correspondence may utilize ray-tracking.”;
PNG
media_image2.png
1706
993
media_image2.png
Greyscale
“Locate POI
[0093] An aspect of the system is to provide a method for locating an area in a query image and matching an annotated area—a POI—in a reference image.”;
“[0125] For example, processor 102 may be integrated into an onboard vehicle computing system, a smartphone or networked to a remote location. Cameras 103 may be vehicle mounts or integrated into a vehicle or in a wearable device like smart glasses. User interface device 106 may be vehicle-mounted or part of a connected smart device, local or remote.”;
“[0127] Camera(s) 103 is typically configured to obtain images of a route traveled, by a vehicle or otherwise. A vehicle may be a mobile or a moving device such as a robot, or any other traveling beings or devices such as a car, boat, or electric bicycles. Routes traveled by a vehicle may include routes in an urban environment or other outdoor or in-door environment. Thus, in one embodiment, a camera 103 may be placed and/or fixed to e.g., glasses worn by a person or to a vehicle, such that at least part of the route being traveled by the vehicle is within the field of view (FOV) of the camera 103.”;
“[0128] Camera 103 is an image capture device and may include a CCD or CMOS or other appropriate chip and a suitable optical system. The camera 103 may be a 2D or 3D camera. In some embodiments the camera 103 may include a standard camera provided, for example, with mobile devices such as smart-phones or tablets.”;
Fig. 4 (shown below): mainly 407;
PNG
media_image3.png
817
1000
media_image3.png
Greyscale
capture image using camera mounted on moving entity and analyze to classify objects in the captured image);
select a comparison target image from among the comparison target image candidates based on at least one of types of the classes included in the captured image, sizes of regions of the respective classes, and proportions of the regions of the respective classes with respect to the entire image, and at least one of types of classes included in each of the comparison target image candidates, sizes of regions of the respective classes, and proportions of the regions of the respective classes with respect to the entire image (Spiegel1:
paras 70-72 (as referenced above and below);
“[0072] One process for characterizing an image is to use feature extraction. The system may be connected to a database containing reference image extractions feature maps and a neural network specifically trained to identify correspondence between a query input and a reference entry in a database. The correspondence may utilize ray-tracking.”;
Paras 75-78 (shown above): mainly paras 76 and 78;
“[0080] According to a feature, a best match for an image may be identified when similarity between the image under consideration is measured against other images. The image in the sequence which has the highest level of similarity may be considered to be the best match, even if not an exact match.”;
PNG
media_image4.png
363
999
media_image4.png
Greyscale
“Locate POI
[0093] An aspect of the system is to provide a method for locating an area in a query image and matching an annotated area—a POI—in a reference image.”
PNG
media_image5.png
1002
993
media_image5.png
Greyscale
“[0129] In one embodiment, processor 102 receives a query image, namely, an image captured by camera 103 whose location is sought. Processor 102 compares the query image to reference images, which are previously captured images, and finds a matching reference image, typically based on similarities between the image and the reference image. The “match” need not be an identical match. The reference image may be matched to a query image based on an acceptable degree of similarity in the absolute sense or relative to other reference images.”;
Fig. 4 (shown above): mainly 408 “CORRESPONDENCE MATCHER”, 409 “OUTPUT CORRESPONDENCE MAP”;
Para 136: “comparing the query image to a set of reference images which includes a sequence of previously obtained images of a route, to find a reference image which matches the query image (step 206).”;
Para 139: comparing using CNN.
“[0179] According to an advantageous embodiment, scene recognition may be accomplished by finding the approximate location of a moving camera, using a database of sparse position-tagged images. A process for self-location is shown in FIG. 5.”;
PNG
media_image6.png
1417
995
media_image6.png
Greyscale
PNG
media_image7.png
1336
797
media_image7.png
Greyscale
“[0183] The set of reference images may include previously captured reference images of the specific route, each reference image corresponding to a known geolocation. It is possible to derive a subset of reference images, near to a known or estimated location which corresponds to the vehicle location.”;
“[0187] Another feature for locating a point of interest (“POI”) is a method for locating an area in a query image and matching an annotated area in a reference image. The location of a POI measures the match to the reference image and may operate to reduce the amount of an image processed to an area of interest.
[0188] The use of traps may be either for a whole reference image or on areas of interest of the reference image. The traps may represent POIs appearing in an image, for example, billboards, shops, buildings, windows, etc.”;
select reference image from among reference image candidates based on: types of classes, sizes of class regions, or proportion class regions relative to entire image – for both the captured image and reference image candidates).
Spiegel1 does not teach extract comparison target image candidates for comparison with the captured image from an information storage based on imaging information of the captured image, the information storage being configured to store one or more past images captured in the past, and imaging information including at least one of imaging date and time information and weather information at an imaging time for each of the one or more past images.
Spiegel2 teaches extract comparison target image candidates for comparison with the captured image from an information storage based on imaging information of the captured image, the information storage being configured to store one or more past images captured in the past, and imaging information including at least one of imaging date and time information and weather information at an imaging time for each of the one or more past images (Spiegel2:
“[0069] A geolocation may be determined based on a reference set which includes images linked to known ambient conditions. A query image obtained by a user during specific ambient conditions, may be compared to a sequence of images previously obtained during similar ambient conditions, to find a matching image in the sequence and a geolocation of the user is determined based on the matching image. In addition, a geolocation may need to be determined even if there is some change in ambient conditions. It is an object to provide a geolocation system with increased tolerance of changes in ambient conditions.”;
Para 68: “A geotag may not be absolutely accurate. The system allows for low resolution and drift or other inaccuracies, within a limit, in the specified location. A query image compared to a set of reference images of a route or other geotagged reference image database to determine a location of the query image capture. The reference image database may include a sequence of pre-obtained images of a route linked to known geolocations. The geolocation of the system can then be determined based on the location of the query image.”;
“[0072] One process for characterizing an image is to use feature extraction. The system may be connected to a database containing reference image extractions feature maps and a neural network specifically trained to identify correspondence between a query input and a reference entry in a database. The correspondence may utilize ray-tracking.”;
PNG
media_image8.png
955
990
media_image8.png
Greyscale
PNG
media_image9.png
197
995
media_image9.png
Greyscale
Para 126: “databases of reference image sequences and reference images linked to known geolocations and/or known ambient conditions.”;
“[0132] The term “ambient conditions” refers to conditions in the environment which affect the imaged scene. Such conditions may include, for example illumination levels and color in the environment being imaged. These conditions may be influenced, for example, by the season of the year, the time of day or night, the location within the city or other site, the amount of vegetation in the scene being imaged and more. Thus, ambient conditions may include time or location related descriptions. For example, ambient conditions may include conditions such as “summer”, “winter”, “evening”, “city center at noon”, “city center at night”, “countryside at night”, etc.”;
PNG
media_image10.png
831
992
media_image10.png
Greyscale
PNG
media_image11.png
564
1504
media_image11.png
Greyscale
PNG
media_image12.png
505
994
media_image12.png
Greyscale
PNG
media_image13.png
605
660
media_image13.png
Greyscale
Para 153 (shown above): “The ambient conditions may be used to further limit the search for a matching reference image”;
“[0157] Ambient conditions are conditions in the environment that affect the imaged scene. Ambient conditions may include time or location related descriptions. For example, ambient conditions may include conditions such as “summer”, “winter””;
“[0173] Typically, in environments where ambient conditions are more significant (e.g., during a snow storm”: a snow storm is definitively a weather event;
PNG
media_image14.png
825
992
media_image14.png
Greyscale
Fig. 3B (shown below): mainly 304;
Fig. 3D (shown below): mainly 312;
PNG
media_image15.png
376
1094
media_image15.png
Greyscale
PNG
media_image16.png
1027
972
media_image16.png
Greyscale
PNG
media_image17.png
609
755
media_image17.png
Greyscale
PNG
media_image18.png
1432
985
media_image18.png
Greyscale
extract reference image candidates from storage based on either: imaging date and time OR weather or both (i.e. “imaging information”)); and
select a comparison target image from among the comparison target image candidates (Spiegel1:
“[0154] A query image may be compared to the sequence of reference images making up set of reference images 20, to find a matching image in the sequence, based on an ambient condition, as further detailed below.”;
Para 155: “The query image I is compared, at processor 102, to a set of reference images in a route (step 304), which can be maintained in storage device 108. The set of reference images may be linked to known ambient conditions, and the query image I is compared to the set of reference images to find a matching reference image. The geolocation of the vehicle (self-location) is determined based on the matching reference image (step 306).”;
select reference image from among reference image candidates)
Thus, it would have been obvious for one of ordinary skill in the art, prior to filing, to implement the teachings of Spiegel2 into Spiegel1 since both Spiegel1 and Spiegel2 suggest a practical solution and field of endeavor of reducing a dense set of reference images to a sparse set of reference images and selecting a reference image from the spares set of reference images in general and Spiegel2 additionally provides teachings that can be incorporated into Spiegel1 in that candidate reference images are first extracted from a fuller set of reference images based on weather and/or date and time (i.e. ambient conditions) as to “The ambient conditions may be used to further limit the search for a matching reference image, thereby reducing the required processing power and providing an improved and facilitated self-localization and mapping device” (emphasis added; Spiegel2: para 153). The teachings of Spiegel2 can be incorporated into Spiegel1 in that candidate reference images are first extracted from a fuller set of reference images based on weather and/or date and time (i.e. ambient conditions). Furthermore, one of ordinary skill in the art could have combined the elements as claimed by known methods and, in combination, each component functions the same as it does separately. One of ordinary skill in the art would have recognized that the results of the combination would be predictable.
As per claim 2, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to extract, as the comparison target image candidates, one or more of the past images of which the imaging information is similar to the imaging information of the captured image (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 1 above;
Paras 153-154, 158, 159; Fig. 3B: 304;
extract reference image candidate that has similar imaging information to the captured image).
As per claim 3, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to extract, as the comparison target image candidates, one or more of the past images of which the imaging information is similar to the imaging information of the captured image (the preceding is the same as claim 2), and imaging position information is similar to imaging position information of the captured image (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 1 above;
Para 130, 159; Fig. 3A: 108;
Para 83: “These query-reference pairs support the construction of the cameras' trajectory, or constitute an anchoring system, complementing a drift-prone odometer. Construction of such a trajectory is useful to prediction of the camera capturing query images and selection of limited “nearby” reference images. The enhanced confidence in location as a result of use of a trajectory enables greater computational efficiency by reducing the reference image query image comparisons needed to find the best match.”;
PNG
media_image19.png
682
995
media_image19.png
Greyscale
PNG
media_image20.png
1487
992
media_image20.png
Greyscale
PNG
media_image21.png
605
660
media_image21.png
Greyscale
extract reference image candidate that has similar imaging position information to the captured image).
As per claim 4, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 1, wherein the information storage further stores, for each of the past images, at least one of types of classes, sizes of regions of the respective classes, and proportions of the regions of the respective classes with respect to the entire image, and the at least one processor is configured to execute the instructions to acquire, for each of the comparison target image candidates, at least one of types of classes, sizes of regions of the respective classes, and proportions of the regions of the respective classes with respect to the entire image, from the information storage (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 1 above;
Para 78, 94;
PNG
media_image22.png
452
992
media_image22.png
Greyscale
“[0176] FIG. 4 shows an image-based location recognition system. Reference images and metadata 403 may be provided to a feature extractor 404. The feature extractor processes the reference images and generates a plurality of feature vectors, which collectively use feature maps 402 stored in a reference database 401. The establishment of the reference database may be an offline process, but may be supplemented, augmented, or updated over time utilizing the query images. The reference database 401 may store location references in the form of feature maps 402 describing an image of a particular location.”;
“[0178] The system may be maintained or updated by incorporating feature maps derived from actual images captured in the location recognition process into the reference database 401. A location may undergo changes over time. These changes may occur in small increments. The system may operate to recognize accurately a location which includes a small incremental change over the location when input as a reference. Over time, the location may be altered in an amount that is sufficient to defeat the location recognition based on an early reference image. The process of using query image feedback into the reference database 401 enhances the ability of the database to change over time to match changes in locations.”;
store and acquire the information used in selecting from the reference image candidates).
As per claim 5, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to store, in the information storage unit, the captured image, the imaging information of the captured image, and at least one of types of the classes into which the objects are classified, sizes of regions of the respective classes, and proportions of the regions of the respective classes with respect to the entire image (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 4 above;
Para 138;
Store the imaging information and the information used in selecting from the reference image candidates).
As per claim 6, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to detect an object from the captured image, and classify a region of the detected object into a class corresponding to the object (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 1 above: POI, traps, building, door, edges, lines, features, etc;
Paras 75, 94; Fig. 4: 407;
Detect object in captured image and classify its region).
As per claim 7, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to divide the captured image into a plurality of regions according to classes to which the respective regions belong (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claims 1 and 6 above;
Paras 75, 94;
Divide the captured image into regions according to classes).
As per claim 8, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to compare the types of the classes included in the captured image with the types of the classes included in each of the comparison target image candidates, and select a past image in which the types of the classes match with those included in the captured image in the highest degree, as the comparison target image, from among the comparison target image candidates (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 1 above: the “types of the classes” optional alternative is taught;
Paras 86, 163; Fig. 4: 408;
Select reference image that has highest degree match of class types relative to captured image. Note that this particular comparison criterion is not required if other alternatively listed comparison criteria are taught in claim 1).
As per claim 9, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to compare the sizes of the regions of the respective classes in the captured image with the sizes of the regions of the respective classes in each of the comparison target image candidates, and select a past image in which the sizes of the regions of the respective classes match with those in the captured image in the highest degree, as the comparison target image, from among the comparison target image candidates (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claims 1 and 8 above: the “types of the classes” optional alternative is taught;
Similar logic to claim 8 above, but “types of the classes” is exchanged for “sizes of the regions”.
Select reference image that has highest degree match of class region sizes relative to captured image. Note that this particular comparison criterion is not required if other alternatively listed comparison criteria are taught in claim 1).
As per claim 10, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to compare the proportions of the regions of the respective classes with respect to the entire image in the captured image with the proportions of the regions of the respective classes with respect to the entire image in each of the comparison target image candidates, and select a past image in which the proportions of the regions of the respective classes with respect to the entire image match with those in the captured image in the highest degree, as the comparison target image, from among the comparison target image candidates (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claims 1 and 8 above: the “types of the classes” optional alternative is taught;
Similar logic to claim 8 above, but “types of the classes” is exchanged for “proportions of the regions of the respective classes with respect to the entire image”.
Select reference image that has highest degree match of proportions of the class regions relative to captured image. Note that this particular comparison criterion is not required if other alternatively listed comparison criteria are taught in claim 1).
As per claim 11, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 1, wherein the at least one processor is configured to execute the instructions to compare the captured image with the selected comparison target image (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 1 above;
Paras 80, 86;
Fig. 4: 407-409;
Para 68: “A geotag may not be absolutely accurate. The system allows for low resolution and drift or other inaccuracies, within a limit, in the specified location. A query image compared to a set of reference images of a route or other geotagged reference image database to determine a location of the query image capture. The reference image database may include a sequence of pre-obtained images of a route linked to known geolocations. The geolocation of the system can then be determined based on the location of the query image.”
Para 80: “The best match for a specific reference image may be found by examining images known to be nearby, for example, from around a route or near anticipated geolocations. The knowledge of the location of a POI relative to the current location may be useful for navigating to the POI. The process may also be helpful to identify a missing POI or a significant change in a POI. To the extent that missing or changed POIs are encountered, remedial action may be required. If the missing POI is not the target of some action, there is an opportunity to correct or supplement the reference database. To the extent that it is the target, the operation may need to be canceled or altered.”;
“[0178] The system may be maintained or updated by incorporating feature maps derived from actual images captured in the location recognition process into the reference database 401. A location may undergo changes over time. These changes may occur in small increments. The system may operate to recognize accurately a location which includes a small incremental change over the location when input as a reference. Over time, the location may be altered in an amount that is sufficient to defeat the location recognition based on an early reference image. The process of using query image feedback into the reference database 401 enhances the ability of the database to change over time to match changes in locations.”;
“[0190] The system may include apparatus and a method for detecting changes through using the approximate location of a moving camera.”
Para 191: “If it is determined that there is a good match quality on a path before and after a zone of reduced match quality, it may be inferred that there has been a change to a portion of the path that is not recognized in the reference database. This may be used to generate a report of an area which should be scanned to update the reference database.”;
compare captured image with selected reference image).
As per claim 12, Spiegel1 in view of Spiegel2 teaches the information processing apparatus according to claim 11, wherein the at least one processor is configured to execute the instructions to calculate a difference between the captured image and the selected comparison target image (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 11 above: especially paras 80, 178, 191 (as referenced above);
calculate difference between captured image and selected reference image).
As per claim 13, Spiegel1 in view of Spiegel2 teaches an information processing system comprising: one or more cameras mounted on a mobile object; and the information processing apparatus according to claim 1 (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 1 above.
Fig. 1: 102, 103;
Para 125, 127
Explicitly incorporates camera mounted on moving entity along with the processor, memory, and instructions from claim 1).
As per claim 14, Spiegel1 in view of Spiegel2 teaches the information processing system according to claim 13, wherein the information processing apparatus acquires, from a plurality of mobile objects, a plurality of captured images captured by using cameras mounted on the respective mobile objects, via a network (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 13 above.
Fig. 1 (shown above);
para 125 (referenced above): “processor… networked to a remote location… remote”;
para 126: “remotely”;
Para 128 (referenced above): “mobile devices such as smart-phones or tablets”: plural mobile objects;
Para 82: “autonomous cars”;
Centralized processing apparatus acquires, via a network, images capture from multiple moving entities).
As per claim 15, Spiegel1 in view of Spiegel2 teaches the information processing system according to claim 13, wherein the at least one processor is configured to execute the instructions to extract, as the comparison target image candidates, one or more of the past images of which the imaging information is similar to the imaging information of the captured image (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claims 2 and 13 above;
as per claim 2 above, extract reference image candidate that has similar imaging information to the captured image).
As per claim 16, Spiegel1 in view of Spiegel2 teaches the information processing system according to claim 13, wherein the at least one processor is configured to execute the instructions to extract, as the comparison target image candidates, one or more of the past images of which the imaging information is similar to the imaging information of the captured image (the preceding is the same as claim 15), and imaging position information is similar to imaging position information of the captured image (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claims 3 and 13 above;
as per claim 3 above, extract reference image candidate that has similar imaging position information to the captured image).
As per claim(s) 17, arguments made in rejecting claim(s) 1 are analogous. Spiegel1 in view of Spiegel2 also teaches an information processing method (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 1 above).
As per claim(s) 18, arguments made in rejecting claim(s) 1 are analogous. . Spiegel1 in view of Spiegel2 also teaches a non-transitory computer-readable medium storing a program for causing a computer to execute (Spiegel1 in view of Spiegel2: See arguments and citations offered in rejecting claim 1 above;
Fig. 1; paras 116-126).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Atiba Fitzpatrick whose telephone number is (571) 270-5255. The examiner can normally be reached on M-F 10:00am-6pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on (571) 270-5183. The fax phone number for Atiba Fitzpatrick is (571) 270-6255.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Atiba Fitzpatrick
/ATIBA O FITZPATRICK/
Primary Examiner, Art Unit 2677
Prosecution Insights