Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The United States Patent & Trademark Office appreciates the application that is submitted by the inventor/assignee. The United States Patent & Trademark Office reviewed the following application and has made the following comments below.
Priority
This application claims benefit of foreign priority under 35 U.S.C. 119(a)-(d) of PCT/US2023/067185, filed on 05/18/2023, and U.S. Provisional Application 63/344,303, filed on 05/20/2022.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 11/19/2024 and 12/31/2024 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Objections
Claims 5, 6, 14, 15, 18, 25 and 26 are objected to because of the following informalities:
Claim 5 presently reads “wherein displaying the graphical indicia on each of the one or more vehicles comprising displaying bounding boxes.” The Examiner recommends replacing the word “comprising” with “comprises.” For example: “wherein displaying the graphical indicia on each of the one or more vehicles comprises displaying bounding boxes.”
Claim 6 presently reads “and wherein the image segmentation generates regions of each obtained images corresponding to the vehicles.” The Examiner recommends replacing the word “images” with “image.” For example: “and wherein the image segmentation generates regions of each obtained image corresponding to the vehicles.”
Claim 14 presently reads “The system of Claim 10 further comprising displaying, via the user interface, a graphical indicia on each of the one or more vehicles to indicate that the vehicle was detected by the system;” The Examiner recommends inserting a comma after “Claim 10” and replacing the “;” at the end of the sentence with a period “.”. Claim 14 is recommended to read: “The system of Claim 10, further comprising displaying, via the user interface, a graphical indicia on each of the one or more vehicles to indicate that the vehicle was detected by the system.”
Claim 15 presently reads “wherein displaying the graphical indicia on each of the one or more vehicles comprising displaying bounding boxes.” The Examiner recommends replacing the word “comprising” with “comprises.” For example: “wherein displaying the graphical indicia on each of the one or more vehicles comprises displaying bounding boxes.”
Claim 18 presently reads “The system of Claim 10 further comprising receiving a updated light indicator receiving a mouse selection from a user which labels the vehicle with the light indicator based on the position of the vehicle.” The Examiner recommends inserting a comma after “Claim 10” and clarifying the section “receiving a updated light indicator receiving a mouse selection.”
Claim 25 presently reads “The method of Claim 21 further comprising… .” The Examiner recommends inserting a comma after “Claim 21.” Claim 25 is recommended to read, “The method of Claim 21, further comprising…”
Claim 26 presently reads “wherein displaying the graphical indicia on each of the one or more vehicles comprising displaying a bounding box around each of the one or more vehicles in the obtained images.” The Examiner recommends replacing the word “comprising” with “comprises.” For example, “wherein displaying the graphical indicia on each of the one or more vehicles comprises displaying bounding boxes.”
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 4-12, 14-16, 18-23, 25-27 and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable over Palefsky-Smith et al. (U.S. Patent No. 10,061,322, hereafter referred to as Palefsky-Smith) in view of Fung et al. (U.S. Patent No. 10,614,326, hereafter referred to as Fung) in further view of Huval (U.S. Patent Pub No. 2018/0373980 A1, hereafter referred to as Huval).
5. Regarding Claim 1, Palefsky-Smith teaches a system for labeling images for training a machine learning model to detect light indicators on a vehicle (Col. 1, lines 62-67 through Col. 2, lines 1-10, Palefsky-Smith teaches a system for controlling a vehicle which includes an image extraction module and a vehicle lighting detection module. The image extraction module accepts sensor data, extracts a plurality of images and a plurality of image labels. The image labels indicate the corresponding lighting state of the observed vehicle in each of the images. A machine learning model is trained utilizing the plurality of images and the plurality of corresponding image labels.), the system including one or more processors and (Col. 5, lines 17-18, Palefsky-Smith teaches at least one processor and a computer-readable storage media.) storing instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising (Col. 5, lines 40-62, Palefsky-Smith teaches the instructions, when executed by the processor 44, receive and process signals from the sensor system 28, perform logic, calculations, methods and/or algorithms for automatically controlling the components of the autonomous vehicle 10.): obtaining images of one or more vehicles on a roadway (Col. 1, lines 62-67, Fig. 8, Palefsky-Smith teaches extracting a plurality of images from sensor data. Fig. 8 shows an example large scale optical image 802 (representing, perhaps the front view from the autonomous vehicle (AV). The image below includes one or more vehicles on a roadway.);
PNG
media_image1.png
256
506
media_image1.png
Greyscale
(Col. 12, lines 37-54, Col. 2, lines 4-6, Col. 11, lines 4-8, Palefsky-Smith teaches label generation module 720 takes as its input sensor data 702, and produces output that comprises labeled images 704. That is, the output includes a set of images of what it has determined are individual observed vehicles, along with a corresponding label (e.g., “brake_lights_on,” “right_turn_signal_on,” or the like). The labels indicate that the corresponding lighting state of the observed vehicle in each of the images. In addition, Palesky-Smith teaches a human operator may be employed to assist in manual interpretation (i.e., labeling training images). For example, when presented with an already cropped and likely correctly-labeled image, the operator merely indicates whether the label is correct.) for a machine learning model
(Abstract, Palefsky-Smith teaches the plurality of images and the plurality of corresponding image labels are later used to train a machine learning model.).
Palefsky-Smith does not explicitly disclose a non- transitory computer storage media and identifying a position of each of the one or more vehicles; displaying, via a user interface, a graphical indicia on each of the one or more vehicles to indicate that the vehicle was detected by the system; and via the user interface,
Fung is in the same field of art of locating and classifying traffic indicators in images for the operation of autonomous vehicles. Further, Fung teaches a non- transitory computer storage media (Col. 2, lines 9-11, Fung teaches a non-transitory computer storage medium.), identifying a position of each of the one or more vehicles (Col. 22, lines 1-6, Col. 12, lines 45-48, Fig. 2F, Fung teaches determining the specific location of the target vehicle 214 with respect to the vehicle 102 (e.g., distance between the front portion of the vehicle 102 and a rear portion of a classified target vehicle 214). Specifically, YOLO detection algorithm predicts class probabilities and bounding box coordinates for objects in the image (see Fig. 2F, bounding box “238”, bounding box identifies the position of the target vehicle 214). Under Broadest Reasonable Interpretation (BRI), the Examiner interprets “one or more” to mean identifying a position of only one vehicle is required to meet the limitation.); displaying, (Col. 15, lines 58-67 through Col. 16, lines 1-3, Fig. 2F, Fung teaches the neural network processing unit 124 computing bounding boxes 236-254 around the objects captured within the image 228. The computed bounding boxes 236-254 encompass the areas of the image 236 that include grid cells that include one [AltContent: arrow]or more of the objects that can include the target vehicle 214 (see bounding box 238, which encloses target vehicle 214, Fig. 2F), additional objects include traffic indicators 216a-216d, 204, 206, 208, 210, etc.);
PNG
media_image2.png
436
459
media_image2.png
Greyscale
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith by identifying the positions/coordinates of each of the one or more vehicles in the captured images and placing bounding boxes around the identified vehicles that is taught by Fung, to make the invention that performs object detection on a captured image to identify vehicles and localize traffic indicators with respect to a vehicle; thus, one of ordinary skilled in the art would be motivated to combine the references to accurately distinguish various features of a driving scene (by localizing them using bounding boxes) to determine when and what type of control is required for operating the autonomous vehicle such as braking, stopping, etc. Additionally, different types of traffic indicators must be distinguished (e.g., traffic light, vehicle brake lights) for controlling the AV (Fung, Col. 1, lines 15-28).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Palefsky-Smith in view of Fung does not explicitly disclose displaying, via a user interface, via the user interface,
Huval is in the same field of art of training a neural network on a training set to identify objects in optical images recorded by a road vehicle. Further, Huval teaches displaying, via a user interface (Paragraph [0085], Huval teaches a remote computer system can interface with multiple instances of the annotation portal to serve (display) optical images containing label conflicts to other human annotators and to collect verification data from these other human annotators.), a graphical indicia on each of the one or more vehicles to indicate that the vehicle was detected by the system (Paragraphs [0031-34], [0037], Figs. 1 & 3, Huval teaches using a neural network to label objects represented in optical images prior to sending these optical images to human annotators for manual labeling.)
PNG
media_image3.png
180
264
media_image3.png
Greyscale
PNG
media_image4.png
303
365
media_image4.png
Greyscale
and receiving, via the user interface, an indication (Paragraphs [0106], [0043], Huval teaches the remote computing system can serve the new video feed to an annotation portal executing on a local computer system for manual labeling by a human annotator; and then receive a manual navigational label attributed to the new video feed by the human annotator. The annotation portal can render a virtual bounding box linked to a cursor and define a geometry associated with the first type of the first manual label.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith in view of Fung by displaying the detected objects (vehicles, light indicators) and light indicator statuses (activated/off) on a user interface that is taught by Huval, to make the invention that displays the automated detection label identified by the neural network/model to a human annotator for confirmation to determine whether the automated label is correct (correctly identified) or incorrect; thus, one of ordinary skilled in the art would be motivated to combine the references to maintain high label quality in order to assemble a large and accurate training set sufficient to train an effective and accurate neural network at reduced cost (Huval, Paragraphs [0012], [0014]). Also, many drivers do not properly user turn signals, etc. which introduces noise into the data. If the noise is above a predetermined threshold, a human operator may be employed to assist in human interpretation (labeling) to indicate whether or not the identified label is correct (Palefsky-Smith, Col. 10, lines 66-67 through Col. 11, lines 1-8).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 2, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 1, wherein obtaining images comprises obtaining images from a plurality of vehicles having autonomous driving systems (Col. 1, lines 62-67, Col. 3, lines 30-34, Palefsky-Smith teaches accepting sensor data associated with the operation or one or more vehicles and extracting a plurality of images from the sensor data.).
In regards to Claim 4, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 1, wherein identifying the position of each of the one or more vehicles comprises identifying vehicles in the images (Col. 10, lines 5-10, Fung teaches YOLO object detection can be utilized to classify target vehicle 214 and localize the target vehicle as being located directly in front of the vehicle 102.) and determining graphical coordinates of the vehicles in the images (Col. 16, lines 29-33, Fung teaches providing localization data with locational coordinates of one or more objects within the images with respect to a location and position of the vehicle 102.).
In regards to Claim 5, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 1, wherein displaying the graphical indicia on each of the one or more vehicles comprising displaying a bounding box around each of the one or more vehicles in the obtained images (Col. 15, lines 58-67 through Col. 16, lines 1-3, Fig. 2F, Fung teaches computing bounding boxes 236-254 around the objects captured within the image 228. The bounding boxes can encompass areas of the images that include grid cells that include one or more of the objects, such as the target vehicle 214. See bounding box “238” in Fig. 2F.).
In regards to Claim 6, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 1, wherein identifying the position of each of the one or more vehicles comprises performing image segmentation on the obtained images (Col. 18, lines 10-22, Fig. 5C, Fung teaches performing image segmentation on the images.), and wherein the image segmentation generates regions of each obtained images corresponding to the vehicles (Col. 18, lines 10-34, Fung teaches performing image segmentation on the red color components using adaptive thresholding. The red color components are converted to a binary image using segmentation based on light intensity. Then, edge detection is performed on the binary image. Based on the edge detection performed on the binary image, “closed shapes” are identified. The closed shapes can be identified as having a closed shape of a classified light indicator (e.g., brake lights). The resulting segmented binary image can be based on light intensity segmentation that contains the identified closed shapes (e.g., brake lights) with irradiating areas in the binary image. The Examiner interprets that brake lights are lights on the vehicle, and therefore identifying the brake lights on the vehicle via segmentation is generating “regions” corresponding to the vehicles.).
In regards to Claim 7, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 1, wherein receiving an indication of whether a light indicator is active or inactive (Col. 10, lines 66-67 through Col. 11, lines 1-8, Abstract, Palefsky-Smith teaches if the noise is above some predetermined threshold, a human operator may be employed to assist in interpretation (i.e., labeling of training images). For example, when presented with an already cropped and likely correctly-labeled image, the operator indicates whether the label is correct. The image labels indicate the corresponding lighting state of the observed vehicles in the image.) comprises receiving a mouse selection from a user (Paragraphs [0036], [0043-45], [0100], Huval teaches providing optical images to a human annotator for insertion of a manual label and/or manual confirmation of an automated label, such as through an annotation portal. The annotation portal can render a virtual bounding box linked to a cursor and defining a geometry associated with the first type of manual label. The annotation portal can host a set of object types related to localization, such as traffic signals. Video feeds can also be labeled with actions (e.g., turning, braking) performed by the road vehicle while the video feed was recorded.) which labels the vehicle as having an active or inactive light indicator (Col. 10, lines 66-67 through Col. 11, lines 1-8, Abstract, Palefsky-Smith teaches if the noise is above some predetermined threshold, a human operator may be employed to assist in interpretation (i.e., labeling of training images). For example, indicating whether the generated label is correct.).
In regards to Claim 8, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 1, wherein receiving the indication of whether a light indicator is active or inactive comprises receiving an indication of whether a brake light is active or inactive (Col. 22, lines 15-30, Fung teaches upon determining the color of the color position of the traffic indicator classified as a brake light is red, the processor can control the vehicle systems to provide an alert to the driver to inform the driver to slow down and stop based on the presence of the target vehicle, the classification of the brake light, the activation of the (red) brake light, and the location of the target vehicle with respect to the vehicle. Alternatively, the processor may autonomously control the vehicle systems to automatically begin slowing down and stopping the vehicle. The Examiner interprets an “alert” to inform the driver to slow down due to an activated (red) brake light to be an indication.).
In regards to Claim 9, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 1, wherein receiving the indication of whether a light indicator is active or inactive comprises receiving an indication of whether a turn signal is active or inactive (Col. 12, lines 37-54, Palefsky-Smith teaches label generation module 720 takes as its input sensor data 702, and produces output that comprises labeled images 704. That is, output includes a set of images of what it has determined are individual observed vehicles, along with corresponding label (e.g., “brake_lights_on,” “right_turn_signal_on,” or the like). The labels indicate that the corresponding lighting state of the observed vehicle in each of the images. The Examiner interprets a “label” corresponding to the image to be an indication.).
In regards to Claim 10, Palefsky-Smith teaches a system for labeling images for training a machine learning model to detect light indicators on a vehicle (Col. 1, lines 62-67 through Col. 2, lines 1-10, Palefsky-Smith teaches a system for controlling a vehicle including an image extraction module and a vehicle lighting detection module. The image extraction module accepts sensor data associated with operation of one or more vehicles; extracts a plurality of images from the sensor data and a plurality of corresponding image labels, wherein the images each include at least a portion of an observed vehicle, and the image labels indicate the corresponding lighting state of the observed vehicle in each of the images; and train a machine learning model utilizing the plurality of images and the plurality of corresponding image labels.) the system including one or more processors and (Col. 5, lines 17-18, Palefsky-Smith teaches at least one processor and a computer-readable storage media.) storing instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising (Col. 5, lines 40-62, Palefsky-Smith teaches the instructions, when executed by the processor 44, receive and process signals from the sensor system 28, perform logic, calculations, methods and/or algorithms for automatically controlling the components of the autonomous vehicle 10.): obtaining images of one or more vehicles on a roadway (Col. 1, lines 62-67, Fig. 8, Palefsky-Smith teaches extracting a plurality of images from sensor data. Fig. 8 shows an example large scale optical image 802 (representing, perhaps the front view from the autonomous vehicle (AV).); (Col. 12, lines 37-54, Col. 9, lines 6-11, Palefsky-Smith teaches an image extraction and label generation module 720 which takes as its input sensor data 702, and produces an output that comprises labeled images 704. That is, output includes a set of images of what it has determined are individual observed vehicles, along with a corresponding label (e.g., “brake_lights_on,” “right_turn_signal_on”, or the like). The system (100) of Fig. 1 determines the lighting state of other vehicles in the environment utilizing a machine learning model that has been trained by automatically extracting and labeling images from sensor data previously acquired.); determining, from the images of one or more vehicles, one or more vehicles having a false prediction of whether the light indicator was active or inactive (Col. 10, lines 66-67 through Col. 11, lines 1-8, Abstract, Palefsky-Smith teaches a human operator may be employed to assist in interpretation (i.e., labeling training images). For example, when presented with an already cropped and likely correctly-labeled image, the operator indicates whether the label is correct. The image labels indicate the corresponding lighting state of the observed vehicle in each of the images. The Examiner interprets indicating whether the label on the image is correct in regards to lighting state labels to be “determining” a false prediction since if the label is determined to be incorrect by the human operator, it is “false.”); and labeling, Col. 10, lines 66-67 through Col. 11, lines 1-8, Abstract, Palefsky-Smith teaches a human operator may be employed to assist in interpretation (labeling of training images) if the noise is above a predetermined threshold. The human operator may be presented with an already cropped and likely correctly-labeled image, the operator indicates whether the label is correct.).
Palefsky-Smith does not explicitly disclose non- transitory computer storage media, identifying a position of each of the one or more vehicles in the obtained images; and labeling, via a user interface, the images having a false prediction
Fung is in the same field of art of locating and classifying traffic indicators in images for autonomous vehicles. Further, Fung teaches a non- transitory computer storage media (Col. 2, lines 9-11, Fung teaches a non-transitory computer storage medium.), and identifying a position of each of the one or more vehicles (Col. 22, lines 1-6, Col. 12, lines 45-48, Fig. 2F, Fung teaches determining the specific location of the target vehicle 214 with respect to the vehicle 102 (e.g., distance between the front portion of the vehicle 102 and a rear portion of a classified target vehicle 214). A YOLO detection algorithm predicts class probabilities and bounding box coordinates for objects in the image (see Fig. 2F, bounding box “238”, bounding box identifies the position of the target vehicle 214). Under Broadest Reasonable Interpretation (BRI), the Examiner interprets “one or more” to mean identifying a position of only one vehicle is required to meet the limitation.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith by using an object detection network to predict classifications and bounding box coordinates of objects in the captured images that is taught by Fung, to make the invention that first localizes the vehicle and the traffic indicator with respect to the vehicle prior to determining the color and/or state of the traffic indicator; thus, one of ordinary skilled in the art would be motivated to combine the references since classifying and localizing driving scene features such as traffic indicators and vehicles is essential for safely controlling the autonomous vehicle. For example, the processor can control the AV to automatically slow or stop the vehicle based on the presence of the target vehicle, classification of the brake light, activation of the brake light, and the location of the target vehicle with respect to the vehicle (Fung, Col. 22, lines 23-30).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Pelfsky-Smith in view of Fung does not explicitly disclose labeling, via a user interface, the images having a false prediction with a correct indication
Huval is in the same field of art of training a neural network on a training set to identify objects in optical images recorded by a road vehicle. Further, Huval teaches labeling, via a user interface, the images having a false prediction with a correct indication (Abstract, Paragraphs [0036-37], [0092], Huval teaches in response to the manual label differing from the automated label, serving the optical image to a human annotator for manual confirmation of one of the manual label and the automated label. The remote computing system provides optical images to a human annotator for insertion of a manual label and/or a confirmation of an automated label through an annotation portal executing on a local computer system (e.g., desktop computer). In particular, the remote computer system can implement this process to detect hard negatives in objects identified by the neural network and to correct the training set with selective, strategic injection of additional human supervision.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith in view of Fung by providing the falsely predicted light state labels to a human annotator for confirmation and/or manual labeling that is taught by Huval, to make the invention that identifies potentially low-quality (e.g., incorrect) labels and sends them to be confirmed/corrected by a human annotator before adding the labeled image to the training set; thus, one of ordinary skilled in the art would be motivated to combine the references to maintain a high label quality in order to develop a larger and more accurate training set sufficient to train an effective and accurate neural network at reduced cost (Huval, Paragraphs [0012], [0014]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 11, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 10, wherein identifying the position of each of the one or more vehicles comprises identifying vehicles in the images (Col. 10, lines 5-10, Fung teaches YOLO object detection can be utilized to classify target vehicle 214 and localize the target vehicle as being located directly in front of the vehicle 102.) and determining graphical coordinates of the vehicles in the images (Col. 16, lines 29-33, Fung teaches providing localization data with locational coordinates of one or more objects within the images with respect to a location and position of the vehicle 102.).
In regards to Claim 12, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 10, wherein obtaining images comprises obtaining images from a plurality of vehicles having autonomous driving systems (Col. 1, lines 62-67, Col. 3, lines 30-34, Palefsky-Smith teaches accepting sensor data associated with the operation or one or more vehicles and extracting a plurality of images from the sensor data.).
In regards to Claim 14, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 10 further comprising displaying, via the user interface, a graphical indicia on each of the one or more vehicles to indicate that the vehicle was detected by the system (Paragraph [0036], Fig. 1, Huval teaches serving the first optical image to an annotation portal executing on a local computer system. The remote computer system provides optical images to a human annotator for insertion of a manual label and/or manual confirmation of an automated label. See automated label in Fig. 1 below “auto.” Also see bounding boxes (labels).).
PNG
media_image5.png
191
305
media_image5.png
Greyscale
In regards to Claim 15, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 14, wherein displaying the graphical indicia on each of the one or more vehicles comprising displaying a bounding box around each of the one or more vehicles in the obtained images (Col. 15, lines 58-67 through Col. 16, lines 1-3, Fig. 2F, Fung teaches computing bounding boxes 236-254 around the objects captured within the image 228. The bounding boxes can encompass areas of the images that include grid cells that include one or more of the objects, such as the target vehicle 214. See bounding box “238” in Fig. 2F.).
In regards to Claim 16, Palefsky-Smith in view of Fung in further view of Huval discloses (Col. 11, lines 58-64, Fig. 1, Palefsky-Smith teaches an autonomous driving system (ADS) 34 within an autonomous vehicle 10. Once the neural network has been trained to identify the “lighting state” of surrounding vehicles, it is provided to one or more vehicles 10. The model can then be used in the ordinary course to detect the lighting state of other vehicles in the vicinity of AV 10, thereby providing AV 10 a tool to predict the behavior of those vehicles.).
In regards to Claim 18, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 10 further comprising receiving a updated light indicator (Col. 10, lines 66-67 through Col. 11, lines 1-8, Abstract, Palefsky-Smith teaches if the noise is above some predetermined threshold, a human operator may be employed to assist in interpretation (i.e., labeling of training images). For example, when presented with an already cropped and likely correctly-labeled image, the operator indicates whether the label is correct. The image labels indicate the corresponding lighting state of the observed vehicles in the image.) receiving a mouse selection from a user which labels the vehicle with the light indicator based on the position of the vehicle (Paragraph [0043], Huval teaches the annotation portal supports insertion of labels onto optical images via placement of labeled boundary boxes around areas of interest within these optical images. For example, the annotation portal can: render a sparse 2D plan LIDAR feed, receive selection of a manual label of a first type, render a virtual bounding box linked to a cursor and defining a geometry associated with the first type of the first manual label, locate the bounding box within the first frame in the sparse 2D LIDAR feed based on a position of cursor input over the first optical image, and label a cluster of points contained within the bounding box as representing the object of the first type. The remote computer system can aggregate these discrete points in the cluster into a first manually-defined region of a first object of the first type in the first frame – representing a field around a road vehicle at a corresponding instant in time. The Examiner interprets “labels the vehicle with the light indicator” to mean the labeled vehicle may have/include a light indicator on it but the light indicators do not specifically need to be labeled.).
In regards to Claim 19, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 10, wherein the indication of whether a light indicator is active or inactive is an indication of whether a brake light is active or inactive (Col. 22, lines 15-30, Fung teaches upon determining the color of the color position of the traffic indicator classified as a brake light is red, the processor can control the vehicle systems to provide an alert to the driver to inform the driver to slow down and stop based on the presence of the target vehicle, the classification of the brake light, the activation of the (red) brake light, and the location of the target vehicle with respect to the vehicle. Alternatively, the processor may autonomously control the vehicle systems to automatically begin slowing down and stopping the vehicle. The Examiner interprets an “alert” to inform the driver to slow down due to an activated (red) brake light to be an indication.).
In regards to Claim 20, Palefsky-Smith in view of Fung in further view of Huval discloses the system of Claim 10, wherein the indication of whether a light indicator is active or inactive is an indication of whether a tum signal is active or inactive (Col. 12, lines 37-54, Palefsky-Smith teaches label generation module 720 takes as its input sensor data 702, and produces output 74 that comprises labeled images 704. That is, output 74 includes a set of images of what it has determined are individual observed vehicles, along with corresponding label (e.g., “brake_lights_on,” “right_turn_signal_on” or the like). The labels indicate that the corresponding lighting state of the observed vehicle in each of the images. The Examiner interprets a label to be an indication.).
In regards to Claim 21, Palefsky-Smith discloses a method for labeling images for training a machine learning model to detect light indicators on a vehicle (Abstract, Col. 12, lines 37-54, Palefsky-Smith teaches a vehicle lighting detection method. Specifically, an image extraction and label generation module takes in sensor data and produces an output that comprises labeled images. The output includes a set of images of what it has determined are individual observed vehicles, along with a corresponding label (e.g., “brake_lights_on,” “right_turn_signal_on”, or the like), which can then be used to train CNN.), the method comprising: obtaining images of one or more vehicles on a roadway (Col. 1, lines 62-67, Fig. 8, Palefsky-Smith teaches extracting a plurality of images from sensor data. Fig. 8 shows an example large scale optical image 802 (representing, perhaps the front view from the autonomous vehicle (AV).); (Col. 12, lines 37-54, Col. 10, lines 66-67 through Col. 11, lines 1-8, Palefsky-Smith teaches an image extraction and label generation module which takes in sensor data and produces an output that comprises labeled images. The output includes a set of images of what it has determined are individual observed vehicles, along with a corresponding label (e.g., “brake_lights_on,” “right_turn_signal_on”, or the like), which can then be used to train CNN. A human annotator may also assist in interpretation (labeling training images.).); determining, from the images of one or more vehicles, one or more vehicles having a false prediction (Col. 10, lines 66-67 through Col. 11, lines 1-8, Abstract, Palefsky-Smith teaches a human operator may be employed to assist in interpretation (i.e., labeling training images). For example, when presented with an already cropped and likely correctly-labeled image, the operator indicates whether the label is correct. The image labels indicate the corresponding lighting state of the observed vehicle in each of the images. The Examiner interprets indicating whether the label on the image is correct in regards to lighting state labels to be “determining” a vehicle having a false prediction since if the label is determined to be not correct by the human operator, it is incorrect or “false.”); and receiving an updated indication of whether the light indicator is active or inactive on the vehicles (Col. 10, lines 66-67 through Col. 11, lines 1-8, Abstract, Palefsky-Smith teaches a human operator may be employed to label training images. For example, the human operator may indicate whether the label is correct when presented with an already cropped and likely correctly labeled image.).
Palefsky-Smith does not explicitly disclose identifying a position of each of the one or more vehicles; labeling, via a user interface, having the false prediction.
Fung is in the same field of art of locating and classifying traffic indicators in images for autonomous vehicles. Further, Fung teaches identifying a position of each of the one or more vehicles (Col. 22, lines 1-6, Fung teaches determining the specific location of the target vehicle with respect to the vehicle 102 (e.g., distance between the front portion of the vehicle 102 and a rear portion of a classified target vehicle). Under Broadest Reasonable Interpretation (BRI), the Examiner interprets “one or more” to mean only one vehicle is required to meet the limitation.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith by using an object detection network to predict classifications and bounding box coordinates of objects in the captured images that is taught by Fung, to make the invention that first localizes the vehicle and the traffic indicator with respect to the vehicle prior to determining the color/state of the traffic indicator; thus, one of ordinary skilled in the art would be motivated to combine the references since classifying and localizing driving scene features such as traffic indicators and vehicles is essential for safely controlling the autonomous vehicle. For example, the processor can control the AV to automatically slow or stop the vehicle based on the presence of the target vehicle, classification of the brake light, activation of the brake light, and the location of the target vehicle with respect to the vehicle (Fung, Col. 22, lines 23-30).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Palefsky-Smith in view of Fung does not explicitly disclose labeling, via a user interface, having the false prediction.
Huval is in the same field of art of training a neural network on a training set to identify objects in optical images recorded by a road vehicle. Further, Huval discloses labeling, via a user interface, (Paragraph [0012], Huval teaches a computer system can serve an optical image to a human annotator via a local computer system and collect a manual label for the optical image from the human annotator via an annotation portal.); and receiving an updated indicationhaving the false prediction (Paragraph [0052], Huval teaches the annotation portal can present a pre-generated automated label for an object detected at an automatically-defined location by the neural network. The human annotator can then reject the automated label and replace the automated label with a different type in a manually-defined location.).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith in view of Fung by providing the falsely predicted light state labels to a human annotator for confirmation and/or manual labeling that is taught by Huval, to make the invention that identifies potentially low-quality (e.g., incorrect) labels and sends them to be confirmed/corrected by a human annotator before adding the labeled image to the training set; thus, one of ordinary skilled in the art would be motivated to combine the references to maintain a high label quality in order to develop a larger and more accurate training set sufficient to train an effective and accurate neural network at reduced cost (Huval, Paragraphs [0012], [0014]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 22, Palefsky-Smith in view of Fung in further view of Huval discloses the method of Claim 21, wherein identifying the position of each of the one or more vehicles comprises identifying vehicles in the images (Col. 10, lines 5-10, Fung teaches YOLO object detection can be utilized to classify target vehicle 214 and localize the target vehicle as being located directly in front of the vehicle 102.) and determining graphical coordinates of the vehicles in the images (Col. 16, lines 29-33, Fung teaches providing localization data with locational coordinates of one or more objects within the images with respect to a location and position of the vehicle 102.).
In regards to Claim 23, Palefsky-Smith in view of Fung in further view of Huval discloses the method of Claim 21, wherein obtaining images comprises obtaining images from a plurality of vehicles having autonomous driving systems (Col. 1, lines 62-67, Col. 3, lines 30-34, Palefsky-Smith teaches accepting sensor data associated with the operation or one or more vehicles and extracting a plurality of images from the sensor data.).
In regards to Claim 25, Palefsky-Smith in view of Fung in further view of Huval discloses the method of Claim 21 further comprising displaying a graphical indicia on each of the one or more vehicles to indicate that the vehicle was detected by the machine leaming model (Col. 15, lines 58-67 through Col. 16, lines 1-3, Fig. 2F, Fung teaches the neural network processing unit 124 computing bounding boxes 236-254 around the objects captured within the image 228. The computed bounding boxes 236-254 encompass the areas of the image 236 that include grid cells that include one or more of the objects that can include the target vehicle 214 (see bounding box 238, Fig. 2F), traffic indicators 216a-216d, 204, 206, 208, 210, etc.).
In regards to Claim 26, Palefsky-Smith in view of Fung in further view of Huval discloses the method of Claim 25, wherein displaying the graphical indicia on each of the one or more vehicles comprising displaying a bounding box around each of the one or more vehicles in the obtained images (Col. 15, lines 58-67 through Col. 16, lines 1-3, Fig. 2F, Fung teaches computing bounding boxes 236-254 around the objects captured within the image 228. The bounding boxes can encompass areas of the images that include grid cells that include one or more of the objects, such as the target vehicle 214. See bounding box “238” in Fig. 2F.).
In regards to Claim 27, Palefsky-Smith in view of Fung in further view of Huval discloses the method of Claim 21, wherein the indication of whether the light indicator is active or inactive of each of the one or more vehicles is predicted by an autonomous driving system of each of the vehicles (Col. 11, lines 58-64, Fig. 1, Palefsky-Smith teaches an autonomous driving system (ADS) 34 within an autonomous vehicle 10. Once the neural network has been trained to identify the “lighting state” of surrounding vehicles, it is provided to one or more vehicles 10. The model can then be used in the ordinary course to detect the lighting state of other vehicles in the vicinity of AV 10, thereby providing AV 10 a tool to predict the behavior of those vehicles.).
In regards to Claim 29, Palefsky-Smith in view of Fung in further view of Huval discloses the method of Claim 21, wherein receiving the updated light indicator (Col. 10, lines 66-67 through Col. 11, lines 1-8, Abstract, Palefsky-Smith teaches if the noise is above some predetermined threshold, a human operator may be employed to assist in interpretation (i.e., labeling of training images). For example, when presented with an already cropped and likely correctly-labeled image, the operator indicates whether the label is correct. The image labels indicate the corresponding lighting state of the observed vehicles in the image.) comprises receiving a mouse selection from a user which labels the vehicle with the light indicator based on the position of the vehicle (Paragraph [0043], Huval teaches the annotation portal supports insertion of labels onto optical images via placement of labeled boundary boxes around areas of interest within these optical images. For example, the annotation portal can: render a sparse 2D plan LIDAR feed, receive selection of a manual label of a first type, render a virtual bounding box linked to a cursor and defining a geometry associated with the first type of the first manual label, locate the bounding box within the first frame in the sparse 2D LIDAR feed based on a position of cursor input over the first optical image, and label a cluster of points contained within the bounding box as representing the object of the first type. The remote computer system can aggregate these discrete points in the cluster into a first manually-defined region of a first object of the first type in the first frame – representing a field around a road vehicle at a corresponding instant in time. The Examiner interprets “labels the vehicle with the light indicator” to mean the labeled vehicle may have/include a light indicator on it but the light indicators do not specifically need to be labeled.).
Claims 3, 13 and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable over Palefsky-Smith et al. (U.S. Patent No. 10,061,322, hereafter referred to as Palefsky-Smith) in view of Fung et al. (U.S. Patent No. 10,614,326, hereafter referred to as Fung) in further view of Huval (U.S. Patent Pub No. 2018/0373980 A1, hereafter referred to as Huval) in further view of Ferguson et al. (U.S. Patent Pub. No. 2020/0033877 A1, hereafter referred to as Ferguson).
Regarding Claim 3, Palefsky-Smith in view of Fung in further view of Huval discloses the system of claim 2.
Palefsky-Smith in view of Fung in further view of Huval does not explicitly disclose wherein obtaining images comprises obtaining images of the plurality of vehicles when the autonomous driving system determines that a light indicator detection was improperly determined by the autonomous driving system.
Ferguson is in the same field of art of identifying objects in an environment of an autonomous vehicle. Further Ferguson teaches wherein obtaining images comprises obtaining images of the plurality of vehicles when the autonomous driving system determines that a light indicator detection was improperly determined by the autonomous driving system (Paragraphs [0071-74], [0076], [0080], Ferguson teaches a vehicle analyzing the data representing objects of the environment to determine at least one object having a detection confidence below a threshold. A processor in a vehicle may be configured to calculate various objects of the environment based on data from various sensors. For example, the processor may detect objects important for an autonomous vehicle to recognize such as pedestrians, street signs, other vehicles, indicator signals on other vehicles, etc. The detection confidence may indicate a likelihood that the determined object is incorrectly identified. The object recognition may also be inconclusive. In response to the detection confidence below a threshold, a subset of the data (image data) is communicated for further processing to a remote operator for further processing. For example, the human operator can correctly identify the object. The Examiner interprets the human operator “obtains images of the plurality of vehicles” (receives image data for further processing/ object confirmation) in response to “when the autonomous driving system determines a light indicator detection was improperly determined” (an incorrect/low confidence detection).).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith in view of Fung in further view of Huval by providing the image data to a human annotator when the object recognition by the autonomous vehicle is inconclusive or has a low confidence value that is taught by Ferguson, to make the invention that provides the subset of images with inconclusive and low confidence object detections to a human operator to correct the object detection; thus, one of ordinary skilled in the art would be motivated to combine the references since detecting objects in the environment can be important to the operation of an autonomous vehicle and the system may request human input when the autonomous system has low confidence in an identification, allowing the vehicle to operate with high precision and affords a safe autonomous operation (Paragraphs [0076], [0024]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 13, Palefsky-Smith in view of Fung in further view of Huval discloses the system of claim 12.
Palefsky-Smith in view of Fung in further view of Huval does not explicitly disclose wherein obtaining images comprises obtaining images of the plurality of vehicles when the autonomous driving system determines that a light indicator detection was improperly determined by the autonomous driving system.
Ferguson is in the same field of art of identifying objects in an environment of an autonomous vehicle. Further Ferguson teaches wherein obtaining images comprises obtaining images of the plurality of vehicles when the autonomous driving system determines that a light indicator detection was improperly determined by the autonomous driving system (Paragraphs [0071-74], [0076], [0080], Ferguson teaches a vehicle analyzing the data representing objects of the environment to determine at least one object having a detection confidence below a threshold. A processor in a vehicle may be configured to calculate various objects of the environment based on data from various sensors. For example, the processor may detect objects important for an autonomous vehicle to recognize such as pedestrians, street signs, other vehicles, indicator signals on other vehicles, etc. The detection confidence may indicate a likelihood that the determined object is incorrectly identified. The object recognition may also be inconclusive. In response to the detection confidence below a threshold, a subset of the data (image data) is communicated for further processing to a remote operator for further processing. For example, the human operator can correctly identify the object. The Examiner interprets the human operator “obtains images of the plurality of vehicles” (receives image data for further processing/ object confirmation) in response to “when the autonomous driving system determines a light indicator detection was improperly determined” (an incorrect/low confidence detection).).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith in view of Fung in further view of Huval by providing the image data to a human annotator when the object recognition by the autonomous vehicle is inconclusive or has a low confidence value that is taught by Ferguson, to make the invention that provides the subset of images with inconclusive/low confidence object detections to a human operator to correct the object detection; thus, one of ordinary skilled in the art would be motivated to combine the references since detecting objects in the environment can be important to the operation of an autonomous vehicle and the system may request human input when the autonomous system has low confidence in an identification, allowing the vehicle to operate with high precision and affords a safe autonomous operation (Paragraphs [0076], [0024]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 24, Palefsky-Smith in view of Fung in further view of Huval discloses the method of claim 21.
Palefsky-Smith in view of Fung in further view of Huval does not explicitly disclose wherein obtaining images comprises obtaining images of the plurality of vehicles when the autonomous driving system determines that a light indicator detection was improperly determined by the autonomous driving system.
Ferguson is in the same field of art of identifying objects in an environment of an autonomous vehicle. Further Ferguson teaches wherein obtaining images comprises obtaining images of the plurality of vehicles when the autonomous driving system determines that a light indicator detection was improperly determined by the autonomous driving system (Paragraphs [0071-74], [0076], [0080], Ferguson teaches a vehicle analyzing the data representing objects of the environment to determine at least one object having a detection confidence below a threshold. A processor in a vehicle may be configured to calculate various objects of the environment based on data from various sensors. For example, the processor may detect objects important for an autonomous vehicle to recognize such as pedestrians, street signs, other vehicles, indicator signals on other vehicles, etc. The detection confidence may indicate a likelihood that the determined object is incorrectly identified. The object recognition may also be inconclusive. In response to the detection confidence below a threshold, a subset of the data (image data) is communicated for further processing to a remote operator for further processing. For example, the human operator can correctly identify the object. The Examiner interprets the human operator “obtains images of the plurality of vehicles” (receives image data for further processing/ object confirmation) in response to “when the autonomous driving system determines a light indicator detection was improperly determined” (an incorrect/low confidence detection).).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith in view of Fung in further view of Huval by providing the image data to a human annotator when the object recognition by the autonomous vehicle is inconclusive or has a low confidence value that is taught by Ferguson, to make the invention that provides the subset of images with inconclusive/low confidence object detections to a human operator to correct the object detection; thus, one of ordinary skilled in the art would be motivated to combine the references since detecting objects in the environment can be important to the operation of an autonomous vehicle and the system may request human input when the autonomous system has low confidence in an identification, allowing the vehicle to operate with high precision and affords a safe autonomous operation (Paragraphs [0076], [0024]).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Claims 17 and 28 are rejected under 35 U.S.C. 103(a) as being unpatentable over Palefsky-Smith et al. (U.S. Patent No. 10,061,322, hereafter referred to as Palefsky-Smith) in view of Fung et al. (U.S. Patent No. 10,614,326, hereafter referred to as Fung) in view of Huval (U.S. Patent Pub No. 2018/0373980 A1, hereafter referred to as Huval) in further view of Smith (U.S. Patent Pub. No. 2009/0174540 A1, hereafter referred to as Smith).
Regarding Claim 17, Palefsky-Smith in view of Fung in further view of Huval discloses the system of claim 10.
Palefsky-Smith in view of Fung in further view of Huval does not explicitly disclose wherein the false prediction is a disagreement between the light indicator and the position of the vehicle.
Smith is in the same field of art of autonomous vehicle operation and determining vehicle intent. Further, Smith discloses wherein the false prediction is a disagreement between the light indicator and the position of the vehicle (Paragraph [0057], Smith teaches the system learns by monitoring whether a turn is indeed made when a turn signal is on. If a turn signal is generated and the vehicle does not turn as indicated by vehicle location and route history, then the system learns it has made a mistake by indicating that a turn did not occur.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith in view of Fung in further view of Huval by detecting a disagreement between the light indication detected and the position of the vehicle that is taught by Smith, to make the invention that detects when the vehicle’s signaled intent and future action do not match; thus, one of ordinary skilled in the art would be motivated to combine the references since many drivers do not properly use turn signals, etc. and in those cases, the vehicle’s signaled (or lack of signaled) intent may not match the vehicle’s actual action (Palefsky-Smith, Col. 10, lines 66-67).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 28, Palefsky-Smith in view of Fung in further view of Huval discloses the method of claim 21.
Smith is in the same field of art of autonomous vehicle operation and determining vehicle intent. Further, Smith discloses wherein the false prediction is a disagreement between the light indicator and the position of the vehicle (Paragraph [0057], Smith teaches the system learns by monitoring whether a turn is indeed made when a turn signal is on. If a turn signal is generated and the vehicle does not turn as indicated by vehicle location and route history, then the system learns it has made a mistake by indicating that a turn did not occur.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Palefsky-Smith in view of Fung in further view of Huval by detecting a disagreement between the light indication detected and the position of the vehicle that is taught by Smith, to make the invention that detects when the vehicle’s signaled intent and future action do not match; thus, one of ordinary skilled in the art would be motivated to combine the references since many drivers do not properly use turn signals, etc. and in those cases, the vehicle’s signaled (or lack of signaled) intent may not match the vehicle’s actual action (Palefsky-Smith, Col. 10, lines 66-67).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kraft et al. (U.S. Patent Pub. No. 2023/0278553 A1) teaches systems and methods of detecting and validating a turn light indicator in a vehicle. A validation system receives a turn light detection signal output from an optical perception system and characterizes the turn light detection signal in terms of frequency and duty cycle and applies filter criteria. If the check determines that the frequency and duty cycle fall within the bounds, then the turn light detection signal is deemed valid. If not, the turn light detection signal is deemed false.
Minster (U.S. Patent Pub. No. 2017/0364758 A1) teaches systems and methods for analyzing vehicle signal lights in order to operate an autonomous vehicle. The method includes receiving an image from a camera regarding a vehicle proximate to the autonomous vehicle. One or more vehicle signals lights of the proximate vehicle is located by the identified camera image as an area of focus.
Xia et al. (U.S. Patent Pub. No. 2026/0141729 A1) teaches a method for improving autonomous driving systems and components by classifying light of vehicles in a driving environment of an autonomous vehicle. The system may perform label adjustments according to label logic to update detected vehicle light labels.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SYDNEY L BLACKSTEN whose telephone number is (571)272-7120. The examiner can normally be reached 8:30am-4:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached at 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SYDNEY L BLACKSTEN/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights