DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4, 8-13, and 16-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Dosovitskiy et al. (CARLA: An Open Urban Driving Simulator, 2017, 1st Conference on Robot Learning, Pages 1-16), hereinafter “Dosovitskiy”.
Regarding claim 1, Dosovitskiy teaches:
A method (See the Abstract.) comprising:
comparing one or more first portions of an image that are associated with an object to one or more second portions of the image that are associated with one or more lanes (See page 4, 1st full paragraph: “In addition to sensor and pseudo-sensor readings, CARLA provides a range of measurements associated with the state of the agent and compliance with traffic rules…Measurements concerning traffic rules include the percentage of the vehicle’s footprint that impinges on wrong-way lanes or sidewalks”. These measurements appear to be image-based, since on page 15, section S.2.3: “The input to the network consists of two most recent images observed by the agent”.);
assigning, based at least on the comparing, the object to a lane of the one or more lanes (See page 16: “Opposite lane: More than 30% of the car’s footprint is over wrong-way lanes.” The examiner considers a detected infraction of “opposite lane” as meeting the claimed “assigning…the object to [an opposite] lane of the one or more lanes”.); and
causing a machine to perform one or more operations based at least on the object being assigned to the lane (See page 4, first full paragraph: “Measurements concerning traffic rules include the percentage of the vehicle’s footprint that impinges on wrong-way lanes or sidewalks, as well as states of the traffic lights and the speed limit at the current location of the vehicle. Finally, CARLA provides access to exact locations and bounding boxes of all dynamic objects in the environment. These signals play an important role in training and evaluating driving policies.” Also see page 6, paragraph before section 5: “Infractions, such as driving on the sidewalk or collisions, do not lead to termination of an episode, but are logged and reported.”).
Regarding claim 2, Dosovitskiy teaches:
The method of claim 1, wherein the assigning the object to the lane comprises: determining, based at least on the comparing, one or more amounts of overlap between the one or more first portions of the image and the one or more second portions of the image; and assigning, based at least on the one or more amounts of overlap, the object to the lane of the one or more lanes (See page 4, 1st full paragraph: “In addition to sensor and pseudo-sensor readings, CARLA provides a range of measurements associated with the state of the agent and compliance with traffic rules…Measurements concerning traffic rules include the percentage of the vehicle’s footprint that impinges on wrong-way lanes or sidewalks”. These measurements appear to be image-based, since on page 15, section S.2.3: “The input to the network consists of two most recent images observed by the agent”.).
Regarding claim 3, Dosovitskiy teaches:
The method of claim 1, wherein the assigning the object to the lane comprises: determining, based at least on the comparing, a first number of pixels associated with the object that overlap with first pixels associated with the lane and a second number of pixels associated with the object that overlap with second pixels associated with a second lane of the one or more lanes; and assigning, based at least on the first number of pixels and the second number of pixels, the object to the lane of the one or more lanes (See page 4, 1st full paragraph: “In addition to sensor and pseudo-sensor readings, CARLA provides a range of measurements associated with the state of the agent and compliance with traffic rules…Measurements concerning traffic rules include the percentage of the vehicle’s footprint that impinges on wrong-way lanes or sidewalks”. The examiner asserts that the impingement is measured in terms of pixels, as evidenced by page 5, “Perception” section: “The perception stack we describe here is built upon a semantic segmentation network based on RefineNet [17]. The network is trained to classify each pixel in the image into one of the following semantic categories: C = {road, sidewalk, lane marking, dynamic object, miscellaneous static}.”).
Regarding claim 4, Dosovitskiy teaches:
The method of claim 3, wherein the assigning the object to the lane is based at least on one or more of: determining that the first number of pixels is greater than the second number of pixels; or determining that the first number of pixels satisfies a threshold number of pixels (See page 16: “Opposite lane: More than 30% of the car’s footprint is over wrong-way lanes.”).
Regarding claim 8, Dosovitskiy teaches:
The method of claim 1, further comprising: determining, based at least on one or more machine learning models processing sensor data, one or more freespace locations; and determining the first portion of the image based at least on the one or more freespace locations (See page 5, “Perception” section: “The probability distributions provided by the network are used to estimate the ego-lane based on the road area and the lane markings. The network output is also used to compute an obstacle mask that aims to encompass pedestrians, vehicles, and other hazards.”).
Regarding claim 9, Dosovitskiy teaches:
The method of claim 1, further comprising determining, based at least on one or more machine learning models processing sensor data, one or more locations associated with the one or more lanes, wherein the one or more second portions of the image correspond to the one or more locations associated with the one or more lanes (See page 15, “Training data” section: “The automated agent has access to privileged information such as locations of dynamic objects, ego-lane, states of traffic lights.”).
Regarding claim 10, Dosovitskiy teaches:
A system comprising (See the Abstract and page 3, 1st full paragraph: “To support this functionality, CARLA is designed as a server-client system, where the server runs the simulation and renders the scene.”):
one or more processors to (See page 3, 1st full paragraph: “To support this functionality, CARLA is designed as a server-client system, where the server runs the simulation and renders the scene.”):
assign, based at least on one or more machine learning models processing sensor data generated using a machine, an object to a lane of one or more lanes associated with an environment (See page 16: “Opposite lane: More than 30% of the car’s footprint is over wrong-way lanes.”); and
causing the machine to perform one or more operations based at least on the object being assigned to the lane (See page 4, first full paragraph: “Measurements concerning traffic rules include the percentage of the vehicle’s footprint that impinges on wrong-way lanes or sidewalks, as well as states of the traffic lights and the speed limit at the current location of the vehicle. Finally, CARLA provides access to exact locations and bounding boxes of all dynamic objects in the environment. These signals play an important role in training and evaluating driving policies.” Also see page 6, paragraph before section 5: “Infractions, such as driving on the sidewalk or collisions, do not lead to termination of an episode, but are logged and reported.”).
Dosovitskiy teaches claim 11 for the reasons given in the treatment of claim 1.
Dosovitskiy teaches claim 12 for the reasons given in the treatment of claim 2.
Dosovitskiy teaches claim 13 for the reasons given in the treatment of claim 3.
Dosovitskiy teaches claim 16 for the reasons given in the treatment of claim 7.
Dosovitskiy teaches claim 17 for the reasons given in the treatment of claim 9.
Regarding claim 18, Dosovitskiy teaches:
The system of claim 10, wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations (See page 4, first full paragraph: “Measurements concerning traffic rules include the percentage of the vehicle’s footprint that impinges on wrong-way lanes or sidewalks…These signals play an important role in training and evaluating driving policies.”); a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.
Regarding claim 19, Dosovitskiy teaches:
One or more processors comprising (See the Abstract and page 3, 1st full paragraph: “To support this functionality, CARLA is designed as a server-client system, where the server runs the simulation and renders the scene.”): processing circuitry to cause a machine to perform one or more operations based at least on one or more object to lane assignments, the one or more object to lane assignments being determined based at least on an overlap between one or more first portions of an image corresponding to objects and one or more second portions of the image corresponding to lanes, the one or more first portions of the image being determined based at least on one or more machine learning models processing sensor data obtained using one or more sensor of the machine (See page 4, 1st full paragraph: “In addition to sensor and pseudo-sensor readings, CARLA provides a range of measurements associated with the state of the agent and compliance with traffic rules…Measurements concerning traffic rules include the percentage of the vehicle’s footprint that impinges on wrong-way lanes or sidewalks”. These measurements appear to be image-based, since on page 15, section S.2.3: “The input to the network consists of two most recent images observed by the agent”. Also see page 16: “Opposite lane: More than 30% of the car’s footprint is over wrong-way lanes.”).
Dosovitskiy teaches claim 20 for the reasons given in the treatment of claim 18.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 5, 6, 14, and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dosovitskiy (CARLA: An Open Urban Driving Simulator, 2017, 1st Conference on Robot Learning, Pages 1-16) in view of Taylor VI (An Advanced Driver-Assistance System: Using Mono-Camera Vision-Based Detection and Tracking for Control of an Autonomous Vehicle, December 2018, Master’s Thesis, Texas A&M University).
Claim 5 is met by the combination of Dosovitskiy and Taylor VI, wherein
Dosovitskiy teaches:
The method of claim 1, further comprising:
Dosovitskiy does not disclose the following; however, Taylor VI discloses:
determining a bounding shape associated with the object; and determining the one or more first portions of the image that are associated with the object by least cropping out a portion of the bounding shape (See the cropping out of the sky and other non-road background portions of Fig. 20 and page 40, 2nd full paragraph: “To remove some of the background information from the detections, attempts were made to crop the image to include only the rear of the car.”).
Dosovitskiy and Taylor VI together disclose the limitations of claim 5. Taylor VI is directed to a similar field of art (vehicle segmentation for control of an autonomous vehicle). Therefore, Dosovitskiy and Taylor VI are combinable. Modifying the system and method of Dosovitskiy by adding the capability of “determining a bounding shape associated with the object; and determining the one or more first portions of the image that are associated with the object by least cropping out a portion of the bounding shape”, as disclosed by Taylor VI, would yield the expected and predictable result of preprocessed image data that excludes irrelevant objects. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine Dosovitskiy and Taylor Vi in this way.
Claim 6 is met by the combination of Dosovitskiy and Taylor VI, wherein
The combination of Dosovitskiy and Taylor Vi discloses:
The method of claim 5, wherein
And Taylor VI further discloses:
the cropping the portion of the bounding shape comprises cropping at least one of: a portion of the bounding shape that is associated with drivable freespace; a portion of the bounding shape that is outside of a drivable surface; or an upper portion of the bounding shape (See the cropping out of the sky and other non-road background portions of Fig. 20 and page 40, 2nd full paragraph: “To remove some of the background information from the detections, attempts were made to crop the image to include only the rear of the car.”).
See the motivation to combine in the treatment of claim 5.
Claim 14 is met by the combination of Dosovitskiy and Taylor VI for the reasons given in the treatment of claim 5.
Claim 15 is met by the combination of Dosovitskiy and Taylor VI for the reasons given in the treatment of claim 6.
Allowable Subject Matter
Claim 7 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The prior art of record, individually or in combination, does not disclose or suggest in claim 7: “determining, based at least on one or more machine learning models processing sensor data, an object fence associated with the object, the object fence indicating a first portion of the object that is associated with a driving surface without indicating a second portion of the object that is outside of the driving surface, wherein the first portion of the image corresponds to the object fence.”
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN S LEE whose telephone number is (571)272-1981. The examiner can normally be reached 11:30 AM - 7:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jonathan S Lee/Primary Examiner, Art Unit 2677
Prosecution Insights