DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities:
Page 3 line 14 states “invention’ ”.
Appropriate correction is required.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 4, 6-7, and 14-15 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Meng US20230162383 (hereinafter “Meng”).
Regarding claim 1, Meng teaches a method for reconstructing a location-dependent scalar additional quantity of a scene from an image of the scene divided into pixels, comprising the following steps (see paragraph 0026 and figure 1, a method for processing an image [which includes pixels] to obtain a depth map [location-dependent scalar additional quantity]):
feeding pixel values of the image to a neural network (see paragraph 0031, a target image is input into a trained relative depth estimation network);
processing each of the pixel values by the neural network to produce respective local difference information, which in each case indicates how a location-dependent scalar additional quantity of the scene changes at a position indicated by the pixel (See paragraphs 0038 and 0053, the determination of ground portion and the relative height as part of the processes to obtain a relative depth map [the relative depth map containing a plurality of pixels for each pixel point in the target image, 0032 and 0067], where the relative height is obtained using a relative depth distance between pixel points for the ground portion and an origin point);
ascertaining scalar additional information for further locations indicated by pixels of the image, from the respective local difference information (see paragraph 0031 and 0038, obtaining a relative depth map of the target image reflecting distance relationships between pixels using the calculated relative depth difference).
Regarding claim 2, Meng teaches the method according to claim 1, wherein the scalar additional quantity includes depth information that is a measure of a distance between a part of the scene and a camera used to record the image (see paragraph 0031, obtain a relative depth map, the relative depth map may reflect a distance relationship between pixels. The process of obtaining relative depth includes the determining the height of the mounted camera, 0066).
Regarding claim 4, Meng teaches the method according to claim 1, wherein at least one reference indication of the location-dependent scalar additional information for at least one position in the scene is additionally included in the ascertainment of the scalar additional information for the further locations (see paragraphs 0036-0038, the use of position information [of ground portion in the target image, 0035] and an origin [reference] in the acquisition of relative depth difference and the relative depth map).
Regarding claim 6, Meng teaches the method according to claim 1, wherein the neural network is additionally configured to ascertain absolute values of the location-dependent scalar additional quantity for the positions indicated by the pixels (see paragraph 0032, obtain an absolute depth map for the target image by converting relative depth of each pixel in the relative depth map).
Regarding claim 7, Meng teaches the method according to claim 6, wherein:
the ascertained absolute values, on the one hand, and the ascertained local difference information, on the other hand, are checked against each other for plausibility (see paragraph 0032, the use of the relative scale to indicate a proportional relationship [plausible] between the relative depth map and the absolute depth map); and
a measure of a reliability of the absolute values and/or the local difference information is ascertained from a result of the plausibility check (see paragraph 0033, a relative scale of the target image for the accuracy of the absolute depth to be obtained).
Claims 14 and 15 are analogous to method claim 1, thus are analyzed and rejected similar to claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 3 and 8 is rejected under 35 U.S.C. 103 as being unpatentable over Meng in view of Shao US 8565513 (hereinafter “Shao”).
Regarding claim 3, Meng teaches the method according to claim 1, wherein the neural network ascertains local difference information based on a reference pixel, the local difference information including changes in the location-dependent scalar additional quantity between the position indicated by the reference pixel, on the one hand, and positions indicated by various (see paragraph 0038, calculating a relative depth difference between a pixel point in the ground portion and an origin [reference point] for a depth map of a target image. The pixel point in the ground potion may be neighboring the origin however it is not explicitly state thus a secondary source will teach obviousness).
Meng does not explicitly teach difference information including changes in the location-dependent scalar additional quantity between the position indicated by the reference pixel and positions indicated by various neighboring pixels.
Shao teaches difference information including changes in the location-dependent scalar additional quantity between the position indicated by the reference pixel and positions indicated by various neighboring pixels (see Eq 1 and 5 and col 5 lines 16-28, determining variation in the horizontal and vertical direction of pixels [including pixel data] through using difference between pixels (i, j) [reference pixel] and (i-1, j) [neighboring pixels]).
Meng and Shao are analogous art because they are from the same field of endeavor of a method of obtaining depth information from an image by comparing pixel values through a difference.
Before the effective filling date of the invention, it would have been obvious to one of ordinary skill in the art to modify Meng to use neighboring pixel to calculating difference of pixels as taught by Shao. The motivation for doing so would have been to determine a variance for each of the input pixel data (Shao, col 4 lines 45-56 and Eq 1 and 5).
Regarding claim 8, Meng teaches the method according to claim 1.
Meng teaches the neural network outputs a quotient of the local difference information and a value of the location-dependent scalar additional quantity at the position indicated by the pixel (see 0031-0032, the trained network obtains the relative depth map and the absolute depth map for each pixel point).
Meng does not teach a quotient of the local difference information and a value of the location-dependent scalar additional quantity.
Shao teaches a quotient of the local difference information and a value of the location-dependent scalar additional quantity (see Eq 11 and 12, the quotient of overall horizontal and vertical variation and the normalized reference value, which is the summation of the variance [a value of the location-dependent scalar additional quantity]).
Claims 5 and 10-11 is rejected under 35 U.S.C. 103 as being unpatentable over Guizilini US 20230334717 (hereinafter “Guizilini”) in view of Meng.
Regarding claim 11, Guizilini teaches a method for training a neural network, comprising the following steps (see paragraph 0015, the training of a neural network):
providing training images of at least one scene (see paragraph 0060, training data for the neural network comprise of training examples with images of a scene);
feeding pixel values of the training images to the neural network to be trained (see paragraph 0060, the use of each pixel of the training image for training the neural network);
processing each of the pixel values by the neural network to produce respective (see paragraph 0061, determine an estimated depth value for each pixel of the training image;
evaluating, using a predetermined cost function, an extent to which the ascertained respective (see paragraph 0064, the use of a loss function based on a difference between the ground truth depth values and the depth values outputted); and
optimizing parameters that characterize a behavior of the neural network, with an aim of improving the evaluation by the cost function (see paragraph 0064-0065, the parameters of the neural network may be optimized to minimize the loss).
Guizilini does not teach local difference information, which in each case indicates how a location-dependent scalar additional quantity of the scene changes at a position indicated by the pixel.
Meng teaches local difference information, which in each case indicates how a location-dependent scalar additional quantity of the scene changes at a position indicated by the pixel (see paragraph 0038, obtaining relative depth difference between pixel points).
Meng and Guizilini are analogous art because they are from the same field of endeavor of a method of obtaining depth information from an image by comparing pixel values through a difference for the use with autonomous vehicles.
Before the effective filling date of the invention, it would have been obvious to one of ordinary skill in the art to modify Guizilini to use local difference information as taught by Meng. The motivation for doing so would have been to acquire a relative scale and height for the relative depth map (Meng, paragraph 0038).
Regarding claim 5, Meng teaches the method according to claim 1.
Meng does not teach a cost function is used to evaluate an extent to which multiple differences of the scalar additional quantity between two predetermined locations in the scene, which differences have been aggregated on different paths from multiple local difference information items, are consistent with each other; and
parameters that characterize a behavior of the neural network are optimized with an aim of improving the evaluation by the cost function.
Guizilini teaches a cost function is used to evaluate an extent to which multiple differences of the scalar additional quantity between two predetermined locations in the scene, which differences have been aggregated on different paths from multiple local difference information items, are consistent with each other (see paragraph 0019, determine a data matching cost between corresponding pixels of images for different depth values); and
parameters that characterize a behavior of the neural network are optimized with an aim of improving the evaluation by the cost function (see paragraph 0064, the parameters of the neural network may be optimized to minimize the loss [the loss is based on the photometric error values, which may be identified as the data matching cost, 0020]).
Regarding claim 10, Meng teaches the method according to claim 1 .
Meng does not teach the image enriched by the ascertained location-dependent scalar additional quantity is included in a planning of the movement of a vehicle, and/or a robot, and/or a production machine configured to pick up and assemble components; and
the vehicle, and/or the robot, and/or the production machine is controlled according to the planning.
Guizilini teaches the image enriched by the ascertained location-dependent scalar additional quantity is included in a planning of the movement of a vehicle, and/or a robot, and/or a production machine configured to pick up and assemble components (see paragraph 0022-0023, the depth map for the image is used make driving decisions); and
the vehicle, and/or the robot, and/or the production machine is controlled according to the planning (see paragraphs 0022-0023, the autonomous vehicle may make driving decisions based in the depth map of the image, which can include driving actions that can be refrained from due to the depth map and the confidence levels).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Meng in view of Vallespi-Gonzales US20170359561 (hereinafter “Vallespi-Gonzales”).
Regarding claim 9, Meng teaches the method according to claim 1.
Meng does not teach the image enriched by the ascertained location-dependent scalar additional quantity is fed to a classifier network, which assigns classification scores with respect to one or more classes of a predetermined classification to the scene and/or at least one object contained in the scene.
Vallespi-Gonzales teaches the image enriched by the ascertained location-dependent scalar additional quantity is fed to a classifier network, which assigns classification scores with respect to one or more classes of a predetermined classification to the scene and/or at least one object contained in the scene (see paragraph 0037 and 0038, a disparity map is inputted into a classifier to classify or identify potential hazards and events which can be scored).
Meng and Vallespi-Gonzales are analogous art because they are from the same field of endeavor of a method of obtaining depth information from an image by comparing pixel values through a difference for use with autonomous vehicles.
Before the effective filling date of the invention, it would have been obvious to one of ordinary skill in the art to modify Meng to use a classifier on the depth to obtain classes as taught by Vallespi-Gonzales. The motivation for doing so would have been to assist an automatic vehicle in maneuvering through road traffic (Vallespi-Gonzales, paragraph 0010).
Allowable Subject Matter
Claims 12-13 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMILY R. HAUK whose telephone number is (571)272-5966. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMILY ROSE HAUK/Examiner, Art Unit 2669
/JOHN B STREGE/Primary Examiner, Art Unit 2669
Prosecution Insights