DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 4 February 2026 have been fully considered but they are not persuasive.
Regarding claims 1, 11 and 20, Applicant argues that Kata fails to teach or suggest “determining, based on the acquired data from all of the multiple cameras of the camera sensor, an operational condition of the SLAM device” because Kato “would generate different statistical results than those of the instant application” (page 9 lines 22-26 of Remarks). However, the Examiner is not persuaded.
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., different statistical results than those of the instant application) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
In response to applicant's arguments against the references individually (page 10 of Remarks), one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Applicant argues that the Williams teaches against individual selection taught by Kato (page 11 of Remarks). However, the examiner respectfully disagrees. Kato teaches at paragraph 143 that the camera with the largest number of feature points is selected and paragraph 71 teaches that more than 1 camera may be selected. Williams teaches using a lookup table to select active cameras (paragraph 62 states “the optimal set of cameras is selected based on a lookup table indicating a next set of cameras to use (e.g., in a particular situation)”. Therefore, the lookup table disclosed by Williams could be used to set the operational cameras based on the feature point data calculated by Kato, where the feature point data corresponds to “a particular situation”. Therefore, the examiner is not persuaded that the references cannot be combined as argued by the applicant.
In view of the foregoing, the claims 1, 11 and 20 stand rejected as further detailed below.
Applicant argues that claims 2-4, 6-10, 12-14 and 16-19 are allowable due to their dependence on an independent claim. In view of the independent claims standing rejected as discussed above, claims 2-4, 6-10, 12-14 and 16-19 also stand rejected as further detailed below.
Regarding claims 5 and 15, Applicant argues that Kato in view of Atwater fails to disclose suing data of “all cameras together” (page 10 of Remarks). However, as discussed above, Kato does use data from all cameras together and therefore this argument is not persuasive for the reasons detailed above.
Regarding claims 5 and 15, Applicant argues that the claims are allowable due to their dependence on an independent claim. In view of the independent claims standing rejected as discussed above, claims 5 and 15 also stand rejected as further detailed below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 6-14 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kato et al. (United States Patent Application Publication 2018/0115711), hereinafter referenced as Kato, in view of Williams et al. (United States Patent Application Publication 20230401732), hereinafter referenced as Williams.
Regarding claim 1, Kato discloses a method for performing camera modality adaptation in a simultaneous localization and mapping (SLAM) device, the SLAM device including a camera sensor comprising multiple cameras (figure 2 exhibits cameras 51 as disclosed at paragraph 95) and a SLAM processor (figure 2 exhibits self-position estimation unit 61 which performs SLAM processing as disclosed at paragraph 164), the method comprising: acquiring data from at least the camera sensor of the SLAM device (paragraph 139 teaches acquiring data including the number of feature points for each camera and a camera index in step S17); determining, based on the acquired data from all the multiple cameras of the camera sensor, an operational condition of the SLAM device (paragraph 139 teaches determining the number of feature points detected by each camera, paragraph 85 teaches that the greater the number of feature points the easier tracking is, therefore a total number of feature points detected by each camera is interpreted as an operational condition); deciding, based on the determined operational condition, a camera modality for the SLAM device, said camera modality being a configuration of the multiple cameras, said configuration defining specific cameras of the multiple cameras as activated cameras and second specific cameras of the multiple cameras as not activated cameras (paragraph 143 teaches selecting the camera with the largest number of feature points to remain active and to put the rest in a sleep state; paragraph 71 teaches that more than one camera may be selected as the activated camera); and controlling, based on the decided camera modality, a camera modality of an image sequence inputted into the SLAM processor (figure 4 exhibits step S63 in which each camera is controlled by changing some cameras to an off state as disclosed at paragraph 144). However, Kato fails to disclose that the configuration is a preset configuration.
Williams is a similar or analogous system to the claimed invention as evidenced Williams teaches a system which dynamically selects cameras wherein the motivation of reducing processing time of the camera by using a table to determine what cameras to control in place of manually determining active and inactive cameras each time would have prompted a predictable variation of Kato by applying Williams’s known principal of providing a look up table with a set of preset configurations which the system uses to control which cameras are active and inactive (paragraph 62 teaches that a camera modality can be select from a look up table which is established prior to image capturing).
In view of the motivations such as reducing processing time of the camera by using a table to determine what cameras to control in place of manually determining active and inactive cameras each one of ordinary skill in the art would have implemented the claimed variation of the prior art system of Kato.
Therefore, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 2, Kato in view of Williams discloses the method of claim 1 as discussed above, in addition, Kato discloses wherein the acquiring step further comprises: acquiring, as the acquired data, data processed inside the SLAM processor (figure 2 shows that feature point detection occurs within self-position estimation unit 61).
Regarding claim 3, Kato in view of Williams discloses the method of claim 1 as discussed above, in addition, Kato discloses wherein the acquiring step further comprises: receiving data outputted from the camera sensor to the SLAM processor, and analyzing the received data to generate the acquired data (paragraph 103 teaches that feature point detection unit 71 analyzes data received from the cameras to generate feature point information).
Regarding claim 4, Kato in view of Williams discloses the method of claim 1 as discussed above, in addition, Kato discloses wherein the determining step further comprises determining, as the operational condition of the SLAM device, a motion tracking difficulty metric of a surrounding environment within which the SLAM device is used, and the motion tracking difficulty metric is evaluated based on at least one of: richness of texture in an image sequence captured by the camera sensor, a number of key points calculated from the image sequence, and a number of feature extracted from the image sequence (paragraph 139 teaches determining the number of feature points detected by each camera, paragraph 85 teaches that the greater the number of feature points the easier tracking is, therefore a total number of feature points detected by each camera is interpreted as a motion tracking difficulty).
Regarding claim 6, Kato in view of Williams discloses the method of claim 1 as discussed above, in addition, Kato discloses wherein the determining step further comprises determining, as the operational condition of the SLAM device, a utilization scenario of the SLAM device, and the utilization scenario includes at least one of: a scale of a room where the SLAM device is used, an intensity of movement by a person wearing the SLAM device, a degree of frame drops in the SLAM device, a degree of camera mis-sync of the camera sensor, a number of moving objects in the room, and an intensity of movement by the moving objects (paragraph 92 teaches that the camera modality may shift to activate all cameras if the movement of the user exceeds an amount).
Regarding claim 7, Kato in view of Williams discloses the method of claim 1 as discussed above, in addition, Kato discloses wherein the deciding step further comprises: upon the determined operational condition meeting a predefined criterion, choosing from a plurality of candidate camera modalities, a camera modality configured with fewer number of cameras compared with other candidate camera modalities (paragraph 173 teaches selecting a number of cameras which detect feature points above a threshold amount as the activation cameras, thus if only a single camera detects a number of feature points greater than the threshold this would be a fewer number of cameras compared to modalities in which two or more cameras detect a number of feature points greater than the threshold).
Regarding claim 8, Kato in view of Williams discloses the method of claim 1 as discussed above, in addition, Kato discloses wherein the controlling step further comprises: selectively activating, based on the decided camera modality, cameras within the camera sensor (figure 4 exhibits step S63 in which each camera is controlled by changing some cameras to an activated state as disclosed at paragraph 144).
Regarding claim 9, Kato in view of Williams discloses the method of claim 1 as discussed above, in addition, Kato discloses wherein the controlling step further comprises: selectively transmitting, based on the decided camera modality, image sequences captured by cameras within the camera sensor to the SLAM processor (figure 4 exhibits step S65 in which only activated cameras transmit image sequences to the processor 61 as disclosed at paragraphs 180 and 147).
Regarding claim 10, Kato in view of Williams discloses the method of claim 1 as discussed above, in addition, Kato discloses wherein the acquiring, determining, deciding, and controlling steps are executed upon the SLAM device being initiated, and/or upon a predefined criterion being met during SLAM operations of the SLAM device (figure 4 exhibits wherein the sleep determination process is carried out if the device is not in sleep mode, step S51, and when the time from a predetermined time exceeds a threshold, step S52 as disclosed at paragraphs 121 and 124).
Regarding claim 11, Kato discloses an apparatus for performing camera modality adaptation in a simultaneous localization and mapping (SLAM) device, the SLAM device including a camera sensor comprising multiple cameras (figure 2 exhibits cameras 51 as disclosed at paragraph 95) and a SLAM processor (figure 2 exhibits self-position estimation unit 61 which performs SLAM processing as disclosed at paragraph 164), the apparatus comprising processing circuitry (figure 2 exhibits activation determination unit 62 as disclosed at paragraph 106) configured to: acquire data from at least the cameras sensor of the SLAM device (paragraph 139 teaches acquiring data including the number of feature points for each camera and a camera index in step S17); determine, based on the acquired data from all the multiple cameras of the camera sensor, an operational condition of the SLAM device (paragraph 139 teaches determining the number of feature points detected by each camera, paragraph 85 teaches that the greater the number of feature points the easier tracking is, therefore a total number of feature points detected by each camera is interpreted as an operational condition); decide, based on the determined operational condition, a camera modality for the SLAM device, the camera modality being a configuration of the multiple cameras, said configuration defining first specific cameras of the multiple cameras as activated cameras and second specific cameras of the multiple cameras as not activated cameras (paragraph 143 teaches selecting the camera with the largest number of feature points to remain active and to put the rest in a sleep state; paragraph 71 teaches that more than one camera may be selected as the activated camera); and control, based on the decided camera modality, a camera modality of an image sequence inputted into the SLAM processor (figure 4 exhibits step S63 in which each camera is controlled by changing some cameras to an off state as disclosed at paragraph 144). However, Kato fails to disclose that the configuration is a preset configuration.
Williams is a similar or analogous system to the claimed invention as evidenced Williams teaches a system which dynamically selects cameras wherein the motivation of reducing processing time of the camera by using a table to determine what cameras to control in place of manually determining active and inactive cameras each time would have prompted a predictable variation of Kato by applying Williams’s known principal of providing a look up table with a set of preset configurations which the system uses to control which cameras are active and inactive (paragraph 62 teaches that a camera modality can be select from a look up table which is established prior to image capturing).
In view of the motivations such as reducing processing time of the camera by using a table to determine what cameras to control in place of manually determining active and inactive cameras each one of ordinary skill in the art would have implemented the claimed variation of the prior art system of Kato.
Therefore, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 12, Kato in view of Williams discloses the apparatus of claim 11 as discussed above, in addition, Kato discloses wherein the processing circuitry is further configured to: acquire, as the acquired data, data processed inside the SLAM processor (figure 2 shows that feature point detection occurs within self-position estimation unit 61).
Regarding claim 13, Kato in view of Williams discloses the apparatus of claim 11 as discussed above, in addition, Kato discloses wherein the processing circuitry is further configured to: receive data outputted from the camera sensor to the SLAM processor, and analyze the received data to generate the acquired data (paragraph 103 teaches that feature point detection unit 71 analyzes data received from the cameras to generate feature point information).
Regarding claim 14, Kato in view of Williams discloses the apparatus of claim 11 as discussed above, in addition, Kato discloses wherein the processing circuitry is further configured to determine, as the operational condition of the SLAM device, a motion tracking difficulty metric of a surrounding environment within which the SLAM device is used, and the motion tracking difficulty metric is evaluated based on at least one of: richness of texture in an image sequence captured by the camera sensor, a number of key points calculated from the image sequence, and a number of feature extracted from the image sequence (paragraph 139 teaches determining the number of feature points detected by each camera, paragraph 85 teaches that the greater the number of feature points the easier tracking is, therefore a total number of feature points detected by each camera is interpreted as a motion tracking difficulty).
Regarding claim 16, Kato in view of Williams discloses the apparatus of claim 11 as discussed above, in addition, Kato discloses wherein the processing circuitry is further configured to determine, as the operational condition of the SLAM device, a utilization scenario of the SLAM device, and the utilization scenario includes at least one of: a scale of a room where the SLAM device is used, an intensity of movement by a person wearing the SLAM device, a degree of frame drops in the SLAM device, a degree of camera mis-sync of the camera sensor, a number of moving objects in the room, and an intensity of movement by the moving objects (paragraph 92 teaches that the camera modality may shift to activate all cameras if the movement of the user exceeds an amount).
Regarding claim 17, Kato in view of Williams discloses the apparatus of claim 11 as discussed above, in addition, Kato discloses wherein the processing circuitry is further configured to: upon the determined operational condition meeting a predefined criterion, choosing from a plurality of candidate camera modalities, a camera modality configured with fewer number of cameras compared with other candidate camera modalities (paragraph 173 teaches selecting a number of cameras which detect feature points above a threshold amount as the activation cameras, thus if only a single camera detects a number of feature points greater than the threshold this would be a fewer number of cameras compared to modalities in which two or more cameras detect a number of feature points greater than the threshold).
Regarding claim 18, Kato in view of Williams discloses the apparatus of claim 11 as discussed above, in addition, Kato discloses wherein the processing circuitry is further configured to: selectively activate, based on the decided camera modality, cameras within the camera sensor (figure 4 exhibits step S63 in which each camera is controlled by changing some cameras to an activated state as disclosed at paragraph 144).
Regarding claim 19, Kato in view of Williams discloses the apparatus of claim 11 as discussed above, in addition, Kato discloses herein the processing circuitry is further configured to: selectively transmit, based on the decided camera modality, image sequences captured by cameras within the camera sensor to the SLAM processor (figure 4 exhibits step S65 in which only activated cameras transmit image sequences to the processor 61 as disclosed at paragraphs 180 and 147).
Claim 20 is a non-transitory computer readable medium including computer readable instructions variant of the method of claim 1 and therefore corresponds to and is analyzed similar to the method of claim 1 (paragraph 389 teaches that recording unit 508 stores a computer program).
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Kato in view of Williams and further in view of Atwater et al. (United States Patent Application Publication 2019/0340440), hereinafter referenced as Atwater.
Regarding claim 5, Kato in view of Williams discloses the method of claim 1 as discussed above, however, Kato fails to disclose wherein the determining step further comprises determining, as the operational condition of the SLAM device, a visual quality of an image sequence captured by the camera sensor, and the visual quality includes at least one of: a level of auto exposure in the image sequence, an amount of motion blur in the image sequence, a level of noise in the image sequence, resolution of the image sequence, and a frames-per-second of the image sequence.
Atwater is a similar or analogous system to the claimed invention as evidenced Atwater teaches an imaging system wherein the motivation of ensuring that images of sufficient quality are obtained thereby improving tracking accuracy would have prompted a predictable variation of Kato by applying Atwater’s known principal of determining a camera modality based on a quality metric which includes camera blur and noise (paragraph 98 teaches that a camera mode and number of activated cameras is maintained if the quality of images is above a threshold amount, paragraph 96 teaches that noise and blurriness are used to determine image quality).
In view of the motivations such as ensuring that images of sufficient quality are obtained thereby improving tracking accuracy one of ordinary skill in the art would have implemented the claimed variation of the prior art system of Kato.
Therefore, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 15, Kato in view of Williams discloses the apparatus of claim 11 as discussed above, however, Kato fails to disclose wherein the processing circuitry is further configured to determine, as the operational condition of the SLAM device, a visual quality of an image sequence captured by the camera sensor, and the visual quality includes at least one of: a level of auto exposure in the image sequence, an amount of motion blur in the image sequence, a level of noise in the image sequence, resolution of the image sequence, and a frames-per-second of the image sequence.
Atwater is a similar or analogous system to the claimed invention as evidenced Atwater teaches an imaging system wherein the motivation of ensuring that images of sufficient quality are obtained thereby improving tracking accuracy would have prompted a predictable variation of Kato by applying Atwater’s known principal of determining a camera modality based on a quality metric which includes camera blur and noise (paragraph 98 teaches that a camera mode and number of activated cameras is maintained if the quality of images is above a threshold amount, paragraph 96 teaches that noise and blurriness are used to determine image quality).
In view of the motivations such as ensuring that images of sufficient quality are obtained thereby improving tracking accuracy one of ordinary skill in the art would have implemented the claimed variation of the prior art system of Kato.
Therefore, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASON A FLOHRE whose telephone number is (571)270-7238. The examiner can normally be reached Mon-Fri 8:00-3:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached at 571-272-7564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
JASON A. FLOHRE
Patent Examiner
Art Unit 2637
/JASON A FLOHRE/Patent Examiner, Art Unit 2637