DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6-8 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites the limitation "the same compute node." There is insufficient antecedent basis for this limitation in the claim. For the purposes of examination, the limitation is interpreted as a software processing unit.
As per claim(s) 7, arguments made in rejecting claim(s) 6 are analogous.
Claim 8 recites the limitations "at least one of the first depth or the second depth." There is insufficient antecedent basis for these limitations in the claim. For the purposes of examination, the first depth and second depth limitations are interpreted as first depth value and second depth value.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental process of computing depth values) without significantly more.
Claim 1 recite(s):
“…receiving a first portion of an image, wherein the first portion is a first resolution”; and “…receiving a second portion of the image, wherein the second portion is a second resolution lower than the first resolution”; can be reasonably interpreted as a human observer viewing and mentally receiving a first portion and second portion of an image.
“…computing, using the first portion, a first depth value within a physical environment”; and “…computing, using the second portion, a second depth value within the physical environment”; Which can be reasonably interpreted as a human observer viewing the portions and mentally computing depth values for objects in the image.
“…adding: the first depth value to a first depth map for the physical environment; and the second depth value to a second depth map for the physical environment, wherein the second depth map is different from the first depth map”; Which can be reasonably interpreted as a human observer, with a pencil and paper, adding the depth values to depth maps.
This judicial exception is not integrated into a practical application because of additional elements:
“…a device”; is a generically recited computer element that does not add a meaningful limitation to the abstract idea because it amounts to simply implementing the abstract idea on a computer and pertains to a generically recited device.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because of additional elements:
“…a device”; is a well-understood, routine, and conventional computer element that does not add a meaningful limitation to the abstract idea because it amounts to simply implementing the abstract idea on a computer and pertains to a well-understood, routine, and conventional device.
Depending claims 2-9 fail to rectify these deficiencies.
Claims 2 & 3 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an additional element of a camera for obtaining the image of claim 1. The camera is recited generically such that it does not provide any meaningful limitations on performing the abstract idea. These claims are not patent eligible.
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of the second portion including the first portion, which is a mental process. The claim is not patent eligible.
Claims 5-7 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of a mathematical calculation, without significantly more. The claims describe the rectification of the image in regards to an order of image processing performed. These claims are not patent eligible.
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to additional elements of a compute node for performing an image analysis operation and a camera for obtaining a second image. The compute node and camera are recited generically such that they do not provide any meaningful limitations on performing the abstract idea. This claim is not patent eligible.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of merging the first and second depth maps to generate a third depth map, which is a mental process. The claim is not patent eligible.
As per claim(s) 10 and 11, arguments made in rejecting claim(s) 1 are analogous. Note that claim 10 recite additional elements: “a non-transitory computer-readable storage medium,” “one or more processors,” and “a device,” which are generically recited and well-understood, routine, and conventional. In addition, claim 11 recites additional elements: “a device”, “one or more processors,” and “memory,” which are generically recited and well-understood, routine, and conventional.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4 and 8-11 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhou et al. (US-20210337175-A1) hereinafter referenced as Zhou.
Regarding claim 1, Zhou discloses: A method, comprising: at a device (Zhou: Abstract): receiving a first portion of an image, wherein the first portion is a first resolution (Zhou: Figures 1 & 12; 0040: “Step 110: Acquire at least two first images, where a resolution of the at least two first images is a first resolution.”; 0121: “The image acquisition unit 310 is configured to: acquire at least two first images, where a resolution of the first images is a first resolution”);
receiving a second portion of the image, wherein the second portion is a second resolution lower than the first resolution (Zhou: Figures 1 & 12; 0043-0044: “Step 120: Acquire at least two second images, where a resolution of the at least two second images is a second resolution, and the second resolution is lower than the first resolution. Optionally, the at least two second images may be acquired by downsampling the at least two first images, respectively.”; 0121: “and acquire at least two second images, where a resolution of the second images is a second resolution, and the second resolution is lower than the first resolution.”; Wherein a second image acquired by down sampling a first image constitutes the second portion of the image.);
computing, using the first portion, a first depth value within a physical environment; computing, using the second portion, a second depth value within the physical environment; and adding: the first depth value to a first depth map for the physical environment; and the second depth value to a second depth map for the physical environment, wherein the second depth map is different from the first depth map (Zhou: 0046: “Step 130 : Use the at least two first images to determine a first depth map corresponding to the at least two first images under the limit of a first disparity threshold.”; 0050: “Step 140: Use the at least two second images to determine a second depth map corresponding to the at least two second images under a limit of a second disparity threshold,”; 0122: “The depth calculation unit 320 is configured to: use the at least two first images to determine a first depth map corresponding to the at least two first images under a limit of a first disparity threshold; and use the at least two second images to determine a second depth map corresponding to the at least two second images under a limit of a second disparity threshold”).
Regarding claim 2, Zhou discloses: The method of claim 1, further comprising: receiving, via a camera, the image at a third resolution, wherein the third resolution is greater than the first resolution (Zhou: 0041: “the at least two first images may be images taken by a binocular camera at the same time, or may be images down-sampled from the images taken by the binocular camera at the same time.”).
Regarding claim 3, Zhou discloses: The method of claim 1, further comprising: receiving, via a camera, the image at the first resolution (Zhou: 0041: “the at least two first images may be images taken by a binocular camera at the same time”).
Regarding claim 4, Zhou discloses: The method of claim 1, wherein the second portion includes the first portion (Zhou: 0043-0044: “Step 120: Acquire at least two second images, where a resolution of the at least two second images is a second resolution, and the second resolution is lower than the first resolution…the at least two second images may be acquired by downsampling the at least two first images”).
Regarding claim 8, Zhou discloses: The method of claim 1, further comprising: performing, by a first compute node based on: at least one of the first depth or the second depth (Claim limitation is interpreted according to interpretation disclosed in the rejection of claim 8 under 35 U.S.C. 112(b)), an image analysis operation using the image, wherein the first compute node is different from a compute node that computed the first depth value and the second depth value (Zhou: 0122-0123: “The depth calculation unit 320 is configured to: use the at least two first images to determine a first depth map corresponding to the at least two first images under a limit of a first disparity threshold; and use the at least two second images to determine a second depth map corresponding to the at least two second images under a limit of a second disparity threshold…The depth combination unit 330 is configured to combine the first depth map and the second depth map to generate a combined depth map.”; Wherein the depth calculation unit and the depth combination unit constitute different compute nodes, and the combination of depth maps generated using a captured image constitutes the image analysis operation.); and an image captured by a camera different from a camera used to capture the image (Zhou: 0004: “Currently, a corresponding depth map may be generated by using a binocular camera system through two images taken by two cameras at two different angles at the same time.”; 0040-0041: “Step 110 : Acquire at least two first images…the at least two first images may originate from a binocular camera.”; Wherein the first and second images are acquired from two different cameras in order to form a depth map.).
Regarding claim 9, Zhou discloses: The method of claim 1, further comprising: merging the first depth map and the second map to generate a third depth map (Zhou: 0123: “The depth combination unit 330 is configured to combine the first depth map and the second depth map to generate a combined depth map.”).
As per claim(s) 10, arguments made in rejecting claim(s) 1 are analogous. In addition, paragraph 0147 of Zhou discloses a non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a device.
As per claim(s) 11, arguments made in rejecting claim(s) 1 are analogous. In addition, paragraph 0147 of Zhou discloses a device, comprising: one or more processors; and memory storing one or more programs configured to be executed by the one or more processors.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou in view of Zhang et al. (US-20230130478-A1) hereinafter referenced as Zhang.
Regarding claim 5, Zhou discloses: The method of claim 1.
Zhou does not disclose expressly: further comprising: rectifying the image before generating the second portion.
Thus, Zhou does not disclose expressly: the rectification of the acquired first image prior to acquiring a second image by down sampling the first image.
Zhang discloses: an image processing pipeline comprising a video image compositor which serves to process images by first performing image rectification, then downscaling the rectified images (Zhang: Figure 3; 0028: “In the example processing pipeline 300 of FIG. 3 , a pair of stereo images is provided as input…the input can be fed to a video image compositor (VIC) 302, which can be a system-on-chip in some embodiments, which can perform image rectification for the pair of images…In this example, each image can also go through downscaling in order to reduce a size or resolution of those images to a smaller, yet corresponding, size or resolution. Any of a number of scaling algorithms may be utilized, as may include a linear filter, nearest-neighbor interpolation, box sampling, mipmap analysis, and the like. Other pre-processing may be performed on the images or image frames as well, as may relate to lens distortion correction in at least one embodiment.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the video image compositor taught by Zhang to process the first image prior to the generation of the second image disclosed by Zhou. The suggestion/motivation for doing so would have been “Rectification can involve projecting the images onto a common image plane, which can help to simplify the recognition of matching objects, points, or features in the respective images.” (Zhang: 0028; Wherein the prior rectifying process reduces the need for the down sampled images to be rectified separately ). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Zhou with Zhang to obtain the invention as specified in claim 5.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou in view of Zhang, and further in view of Moraleda et al. (US-20170178301-A1) hereinafter referenced as Moraleda.
Regarding claim 6, Zhou in view of Zhang discloses: The method of claim 5.
Zhou in view of Zhang does not disclose expressly: further comprising: before rectifying the image, performing an image analysis operation using the image, wherein the image analysis operation is performed by the same compute node that rectified the image (Claim limitation is interpreted according to interpretation disclosed in the rejection of claim 6 under 35 U.S.C. 112(b)).
Moraleda discloses: an image recognition application for performing image rectification on an input image, wherein the application performs an object recognition task and extracts regions of interest from the input image and performs image rectification based on the detected objects and their regions of interest (Moraleda: Figures 16A & 16B; 0096: “FIGS. 16A and 16B are a flow diagram 1600 illustrating another embodiment of a method for performing image rectification on an input image…the image recognition module 203 receives 1602 an input image. At 1604 , the image recognition module 203 identifies a plurality of objects in the input image. The image recognition module 203 also identifies a ROI associated with each object…
the local transformation module 301 computes a location transformation matrix for an object based on mapping the ROI of the object in the input image to the ROI of the object in the indexed image. At 1608 , the parameter estimation module 303 estimates rectification parameters (θ, ψ, φ) based on the local transformation matrix for each of the plurality of objects, wherein (θ, ψ, φ) are used to compute a global transformation matrix for rectifying the input image.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the image recognition model used to identify objects of interest for image rectification taught by Moraleda into the video image compositor disclosed by Zhou in view of Zhang in order to detect and rectify images containing target objects. The suggestion/motivation for doing so would have been “The image recognition module 203 identifies one or more objects in the input image matching the stored features. The image recognition module 203 returns the matched objects and information associated with the matched objects as a recognition result. In some embodiments, the returned information may include a list of object attributes, a spatial location of an object in the received image, and a region of interest (ROI) bordering the object in the received image. For example, the attributes for a product may include manufacturer, size, color, packaging, a product description, material, etc.” (Moraleda: 0053). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Zhou in view of Zhang with Moraleda to obtain the invention as specified in claim 6.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou in view of Zhang, and further in view of Afrooze et al. (US-20230415348-A1) hereinafter referenced as Afrooze.
Regarding claim 7, Zhou in view of Zhang discloses: The method of claim 5.
Zhou in view of Zhang does not disclose expressly: further comprising: after rectifying the image, performing an image analysis operation using the image, wherein the image analysis operation is performed by the same compute node that rectified the image (Claim limitation is interpreted according to interpretation disclosed in the rejection of claim 7 under 35 U.S.C. 112(b)).
Afrooze discloses: an image processing vision pipeline, wherein after rectifying an input image, performs an image analysis operation using the image, wherein the image analysis operation is performed by the same processor that rectified the image (Afrooze: 0075-0076: “Referring now to FIG. 6 , there is shown a first vision pipeline 600 a for object pose estimation using stereoscopic depth estimation, according to an example embodiment… In FIG. 6, the first node 602 a captures the stereo image pair, which comprises defining camera parameters of the image capture (e.g., exposure, gain, bit depth) and performing basic processing (e.g., gamma correction, multi-exposure fusion, underexposure/overexposure adjustment); the second node 602 b performs stereo rectification on the image pair; the third node 602 c crops the region of interest from the image pair; the fourth node 602 d performs object detection on the cropped region of interest; the fifth node 602 e performs stereo depth estimation on the detected object…The vision processor 108 a executes the vision pipeline 600 a in response to a call from the robot controller 110 ,”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the algorithms for cropping the region of interest and performing object detection taught by Afrooze after the image rectification performed by the video image compositor disclosed by Zhou in view of Zhang. The suggestion/motivation for doing so would have been “ the fifth node 602 e performs stereo depth estimation on the detected object” (Afrooze 0076; Wherein the stereo depth estimation is performed only on objects of interest, reducing processing needed.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Zhou in view of Zhang with Afrooze to obtain the invention as specified in claim 7.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J RODRIGUEZ whose telephone number is (703)756-5821. The examiner can normally be reached Monday-Friday 10am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J RODRIGUEZ/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672