DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. Applicant’s response to the Non-final Office Action dated 12/22/2025, filed with the office on 03/19/2026, has been entered and made of record.
Priority
Acknowledgement is made of Applicant’s claim of continuation from International Application No. PCT/JP2021/023999, filed on June 24, 2021.
Information Disclosure Statement
The information disclosure statement (“IDS”) filed on 12/18/2023 has been reviewed and the listed references have been considered.
The information disclosure statement (“IDS”) filed on 04/29/2026 have been reviewed and the listed references have been considered.
Response to Amendment
5. In light of Applicant’s amendment of the title of the invention, the objection of record with respect to the specification has been withdrawn.
In light of Applicant’s amendments of the independent claims, Applicant’s arguments presented in page 8, first paragraph of its Reply has been found to be persuasive. Accordingly, the rejection of record of claims under 35 U.S.C. 101 has been withdrawn.
Response to Arguments
Applicant’s amendment of independent Claim 1 which has altered the scope of the claims of the instant application, has necessitated the new ground(s) of rejection presented in this office action with respect to claims of the instant application. Accordingly, in response to Applicant’s arguments that are merely directed to the amended portion of the claims, new analyses have been presented below, which make Applicant’s arguments moot.
Consequently, THIS ACTION IS MADE FINAL.
Status of Claims
Claims 1, 3-6, 8 and 9 are pending. Claims 1, 3-6, 8 and 9 are amended. Claims 2 and 7 are cancelled.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “first image capturer” and “second image capturer”, in claims 1, 5, 8 and 9.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, these are being interpreted to cover the corresponding structures described in the applicant’s drawings: schematics shown in Fig. 3, algorithms (flow charts) depicted in Fig. 16, and applicant’s specification: ¶0015: “image capturing unit 12 is a digital camera capable of capturing a moving image” as performing the claimed functions, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Specifically, Claim 1 recites “deform the projection plane”. There is insufficient antecedent basis for “the projection plane” in the claims. Which means, the claim does not explicitly recite a projection plane in the claim language.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-5, 8 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Aflalo et al. (US 2019/0188872 A1) in view of Rathi et al. (US 2014/0247352 A1).
Regarding claim 1, Aflalo teaches, An information processing device (Aflalo, ¶0073: “Image processing apparatus 10 may be included as part of an electronic device”) comprising processing circuitry configured to: (Aflalo, ¶0076: “a central processing unit (CPU) and/or other processing circuitry”) acquire first point cloud information based on first image data obtained from a first image capturer provided at a first position (Aflalo, ¶0035: “First camera C.sub.1 may capture a first image of a scene including one or more is objects O. The first image may be represented by a first point cloud, which is provided to an image processing apparatus”) of (Aflalo, ¶0037: “a second point cloud representing the second image is provided to image processing apparatus”) obtained from a second image capturer provided at a second position different from the first position of (Aflalo, ¶0037: “A second depth camera… may be located at a second position x.sub.2, y.sub.2, z.sub.2 corresponding to a second viewpoint V.sub.2, and may capture a second image”) perform alignment processing on the first point cloud information and the second point cloud information; (Aflalo, ¶0015: “performing a depth weighted based alignment of corresponding points of the first and second point clouds”) generate integrated point cloud information by using the first point cloud information and the second point cloud information on both of which the alignment processing has been performed; (Aflalo, ¶0039: “once at least the first and second point cloud images have been captured and aligned using ICP-based processing, image processing apparatus 10 may perform panoramic or other combinatorial image processing to build a database of 3D rendered composite images representing the scene/object”) and perform the alignment processing using the first point cloud information acquired in the first processing and the second point cloud information acquired in the second processing. (Aflalo, ¶0018: “match feature points extracted from a first point cloud with feature points extracted from a second point cloud; determine an initial rotation and translation of the first point cloud with respect to the second point cloud to initially align the first and second point clouds”). However, Aflalo does not explicitly teach, deform the projection plane based on the integrated point cloud information; project images of surroundings of the moving body, including the first image data and the second image data, onto the deformed projection plane; and control a display of the moving body to display images of surroundings of the moving body projected onto the deformed projection plane, wherein the processing circuitry is configured to shift from first processing of acquiring the first point cloud information to second processing of acquiring the second point cloud information, based on a gear change of the moving body.
In an analogous field of endeavor, Rathi teaches, deform the projection plane based on the integrated point cloud information; (Rathi, ¶0038: “correct the virtual (assumed) ground plane for the virtual top view vision projection in the manner the vehicle is rolled or pitched”) project images of surroundings of the moving body, including the first image data and the second image data, onto the deformed projection plane; (Rathi, ¶0016: “image portions taken by the respective cameras, which add to a stitched virtual top view image, shown with objects symbolized by boxes passing the scene surrounding the vehicle when the vehicle is driving forward”) and control a display of the moving body to display images of surroundings of the moving body (Rathi, ¶0003: “Such systems may display images for viewing by the driver of the vehicle that provide a view exterior of the vehicle”) projected onto the deformed projection plane, (Rathi, ¶0048: “displaying the projected path of a vehicle driving behind the subject or equipped vehicle but in another lane”) wherein the processing circuitry is configured to shift from first processing of acquiring the first point cloud information to second processing of acquiring the second point cloud information, based on a gear change of the moving body. (Rathi, ¶0061: “display images captured by a rearward viewing camera of the vehicle during a reversing maneuver of the vehicle (such as responsive to the vehicle gear actuator being placed in a reverse gear position”)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Aflalo using the teachings of Rathi to introduce generating a deformed projection of vehicle surroundings. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of a surrounding view of a moving vehicle for improved driving safety. Therefore, it would have been obvious to combine the analogous arts Aflalo and Rathi to obtain the invention in claim 1.
Regarding claim 3, Aflalo in view of Rathi teaches, The information processing device according to claim 1, wherein the processing circuitry is configured to calculate difference information (Aflalo, ¶0053: “ICP process 108 may generally involve determining rotation and translation differences between the matching feature points”) including at least one of a translational movement amount and a rotational movement amount of at least one of the first point cloud information and the second point cloud information, (Aflalo, ¶0010: “ICP iteratively revises a transformation based on a combination of rotation and translation, to minimize errors in distances between the corresponding points of the first and second point clouds”) and perform the alignment processing based on the obtained difference information. (Aflalo, ¶0010: “Thereby, the reference (first) point cloud, and the coordinate-transformed second point cloud, become substantially aligned”).
Regarding claim 4, Aflalo in view of Rathi teaches, The information processing device according to claim 1, wherein the processing circuitry is configured to shift from the first processing to the second processing further based on at least one of information indicating a change of a traveling direction of the moving body or speed information of the moving body. (Rathi, ¶0061: “display images captured by a rearward viewing camera of the vehicle during a reversing maneuver of the vehicle (such as responsive to the vehicle gear actuator being placed in a reverse gear”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Aflalo in view of Rathi using the additional teachings of Rathi to introduce activating a rear view when the car is in reverse. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of displaying the surroundings related to driving backwards for safer navigation. Therefore, it would have been obvious to combine the analogous arts Aflalo and Rathi to obtain the invention in claim 4.
Regarding claim 5, Aflalo in view of Rathi teaches, The information processing device according to claim 1, wherein the first image capturer and the second image capturer are configured to capture images in mutually different directions with respect to the moving body. (Aflalo, ¶0044: “first and second image frames captured from different viewpoints”).
Regarding claim 8, Aflalo teaches, An information processing method to be performed by a computer, (Aflalo, ¶0044: “computer-implemented image processing method”) the method comprising: acquiring first point cloud information based on first image data obtained from a first image capturer provided at a first position (Aflalo, ¶0035: “First camera C.sub.1 may capture a first image of a scene including one or more is objects O. The first image may be represented by a first point cloud, which is provided to an image processing apparatus”) that acquires second point cloud information based on second image data (Aflalo, ¶0037: “a second point cloud representing the second image is provided to image processing apparatus”) obtained from a second image capturer provided at a second position different from the first position of the (Aflalo, ¶0037: “A second depth camera… may be located at a second position x.sub.2, y.sub.2, z.sub.2 corresponding to a second viewpoint V.sub.2, and may capture a second image”) performing alignment processing on the first point cloud information and the second point cloud information; (Aflalo, ¶0015: “performing a depth weighted based alignment of corresponding points of the first and second point clouds”) and generating integrated point cloud information by using the first point cloud information and the second point cloud information on both of which the alignment processing has been performed. (Aflalo, ¶0039: “once at least the first and second point cloud images have been captured and aligned using ICP-based processing, image processing apparatus 10 may perform panoramic or other combinatorial image processing to build a database of 3D rendered composite images representing the scene/object”). However, Aflalo does not explicitly teach, a moving body.
In an analogous field of endeavor, Rathi teaches, a moving body. (Rathi, ¶0035: “the vehicle is in motion (such as driving forwardly or rearwardly”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Aflalo using the teachings of Rathi to introduce a moving vehicle. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of sensing the surrounding of a moving vehicle for improving driving safety. Therefore, it would have been obvious to combine the analogous arts Aflalo and Rathi to obtain the invention in claim 8.
Regarding claim 9, Aflalo further teaches, A computer program product being information processing program product including programmed instructions embodied in and stored on a non-transitory computer readable medium, wherein the instructions, when performed by a computer, cause the computer to perform: (Aflalo, ¶0075: “computer program instructions may be stored in a non-transitory computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner”) acquiring first point cloud information based on first image data obtained from a first image capturer provided at a first position (Aflalo, ¶0035: “First camera C.sub.1 may capture a first image of a scene including one or more is objects O. The first image may be represented by a first point cloud, which is provided to an image processing apparatus”) of (Aflalo, ¶0037: “a second point cloud representing the second image is provided to image processing apparatus”) obtained from a second image capturer provided at a second position different from the first position of the (Aflalo, ¶0037: “A second depth camera… may be located at a second position x.sub.2, y.sub.2, z.sub.2 corresponding to a second viewpoint V.sub.2, and may capture a second image”) performing alignment processing on the first point cloud information and the second point cloud information; (Aflalo, ¶0015: “performing a depth weighted based alignment of corresponding points of the first and second point clouds”) and generating integrated point cloud information by using the first point cloud information and the second point cloud information on both of which the alignment processing has been performed. (Aflalo, ¶0039: “once at least the first and second point cloud images have been captured and aligned using ICP-based processing, image processing apparatus 10 may perform panoramic or other combinatorial image processing to build a database of 3D rendered composite images representing the scene/object”). However, Aflalo does not explicitly teach, a moving body.
In an analogous field of endeavor, Rathi teaches, a moving body. (Rathi, ¶0035: “the vehicle is in motion (such as driving forwardly or rearwardly”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Aflalo using the teachings of Rathi to introduce a moving vehicle. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of sensing the surrounding of a moving vehicle for improving driving safety. Therefore, it would have been obvious to combine the analogous arts Aflalo and Rathi to obtain the invention in claim 9.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Aflalo et al. (US 2019/0188872 A1) in view of Rathi et al. (US 2014/0247352 A1) and in further view of Bravo Orellana et al. (US 2019/0271780 A1).
Regarding claim 6, Aflalo in view of Rathi teaches, The information processing device according to claim 1. However, the combination of Aflalo and Rathi does not explicitly teach, wherein the processing circuitry is configured to perform first processing of acquiring the first point cloud information and second processing of acquiring the second point cloud information in parallel, and the alignment processing by using the first point cloud information and the second point cloud information acquired in parallel.
In an analogous filed of endeavor, Bravo Orellana teaches, wherein the processing circuitry is configured to perform first processing of acquiring the first point cloud information and second processing of acquiring the second point cloud information in parallel, (Bravo Orellana, ¶0010: “two point clouds simultaneously acquired by the two tridimensional sensors”) and the alignment processing by using the first point cloud information and the second point cloud information acquired in parallel. (Bravo Orellana, ¶0155: “By “simultaneous multi-scans alignment”, it is meant that the newly received point cloud… scans that needs to be aligned together simultaneously”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Aflalo in view of Rathi using the teachings of Bravo Orellana to introduce simultaneously processing and aligning point cloud information. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of accurately and efficiently track an object across a multiple camera system. Therefore, it would have been obvious to combine the analogous arts Aflalo, Rathi and Bravo Orellana to obtain the invention in claim 6.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEHRAZUL ISLAM whose telephone number is (571)270-0489. The examiner can normally be reached Monday-Friday: 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Saini Amandeep can be reached on (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEHRAZUL ISLAM/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights