DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
New claims 16-19 are added herein.
Applicant’s arguments, see remarks, filed 11/11/2025, with respect to the rejection of the claims under 35 U.S.C. 101 have been fully considered and are persuasive, and therefore is withdrawn.
Applicant’s arguments, see remarks, filed 11/11/2025, with respect to the rejection of the claims under 35 U.S.C. 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground of rejection is made in view of U.S. Patent Application Publication No.: 2019/0392242 (Tariq et al.) under 35 U.S.C. 103.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
New claim 19 is rejected under 35 U.S.C. 112(a) as failing to comply with the written description requirement. The claim contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, at the time the application was filed, had possession of the claimed invention. There is no discussion in the specification regarding user’s faces, facial positions, central or optical axes, angles of any sort, or active image sensors. This claim contains new matter. Proper corrections are required.
Regarding new claim 19, the claim depends from claim 1 however shares no subject matter in common with dependent claim 19; both claims are in disparate fields; independent claim 1 is directed to optimizing a biomedical training image dataset for training a machine learning model, to be more precise, while dependent claim 19 is directed to detecting a user facial positions and angles; there is no overlap between the claims regarding subject matter; therefore attempting a rejection of claim 19 under prior art 35 U.S.C. 102 and 103 is practically impossible to do for Examiner and thus will not be examined under prior art.
Claims 1, 11, and 15 are rejected under 35 U.S.C. 112(a) as failing to comply with the written description requirement. The claims contain subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, at the time the application was filed, had possession of the claimed invention. The claims recite using “transfer learning” to update/train the object detection model using the training dataset, however the way transfer learning is used in the claim as well in the specification, is incorrect compared to its term of the art definition within machine learning; transfer learning is defined as a machine learning technique in which knowledge gained through one task or dataset is used to improve model performance on another related task or different dataset; in other words, transfer learning uses what has been learned in one setting to improve generalization in another setting; this is done typically in the field of computer vision, to pre-train a first machine learning model (source) trained on a large dataset of images to do general object detection (ex: any object), and the learned knowledge such as parameters or weights from initial layers of the source model are taken to be used in a second machine learning model (target) for a different, but unrelated detection task (ex: humans); the benefit of transfer learning is leveraging the knowledge from the first, already-developed model to build the second, more specialized one more efficiently, rather than training the target model from scratch; therefore, transfer learning inherently involves the use of two or more machine learning models. This is in contrast to Applicant’s claim and present specification reciting updating/re-training the (same) object detection model using the training dataset, and para. [0058] reciting that “with transfer learning, the original training of the object detection model 106 may be maintained and adjusted with additional training with the training image dataset 118 to correct for weaknesses in the object detection model 106 that result in the misdetection region 114”; this definition provided in the specification is not the generally agreed upon definition of transfer learning by one of ordinary skill in the art in machine learning, and discusses simply updating the training data of a single machine learning model with more accurate training data that incorporates aspects of misdetection, “blind spots”, false positives, and false negatives, into the training data; this is typically better known as model retraining, adaptive retraining, active learning, or hard example mining, rather than transfer learning. Proper Corrections are Requested.
Dependent claims 2-5, 12-14, 16, and 18-19 fail to cure the deficiencies of independent claims 1 and 11, respectively, and thus are also rejected under 35 U.S.C. 112(a) for these reasons.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1, 11, and 15 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, regards as the invention. As discussed in the 35 U.S.C. 112(a) written description rejection above, the term “transfer learning” is indefinite and not used properly as defined by one of ordinary skill in the art of machine learning; with regards to claims 1 and 11, Examiner will be interpreting the claim to recite “update the object detection model using the training image dataset for learning”; with regards to claim 11, Examiner will be interpreting the claim to recite “perform learning to train the object detection model with the training image dataset”. Proper Corrections are Requested.
Dependent claims 2-5, 12-14, 16, and 18-19 fail to cure the deficiencies of independent claims 1 and 11, respectively, and thus are also rejected under 35 U.S.C. 112(b) for these reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 6, and 9-18 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Publication No.: 10,909,349 (Tripathi et al.) (hereinafter Tripathi), in view of U.S. Patent Application Publication No.: 2019/0392242 (Tariq et al.) (hereinafter Tariq).
Regarding claim 1, Tripathi teaches an electronic device, comprising: a processor to: (Tripathi, col. 14, lines 1-3: “FIG. 3 is a block diagram showing an example architecture 300 of a computing device, such as the processors and other computing devices described herein … The processing element 304 may comprise at least one processor.”)
generate an evaluation image dataset to determine precision of a machine learning object detection model; run the evaluation image dataset on the object detection model to designate a misdetection region in the evaluation image dataset; generate a training image dataset to adjust the object detection model based on the identified misdetection region; and update the object detection model by learning using the training image dataset (Tripathi, col. 9, lines 63-67; col. 10, lines 1-32; col. 12, lines 50-67; col. 13, lines 1-30: “As discussed in further detail below, parameters of theta learner 124 (e.g., weights and/or biases) may be updated during processing using back propagation to improve the performance of theta learner 124. The training objective of theta learner 124 may be to maximize misclassification loss 132 of detector 128. Stated another way, the training objective of theta learner 124 is to minimize the confidence score of object-of-interest detections made by detector 128 and thereby to generate “hard positive” synthetic composite image data. Hard positive synthetic composite image data is image data that includes 3D model 150 (as transformed by theta learner 124) composited with background image data 152 in which detector 128 is unable to detect any objects-of-interest—despite the presence of objects-of-interest represented within the synthetic composite image data. Misclassification loss may be loss related to object detection by detector 128. A training objective of theta learner 124 may be to increase misclassification loss of detector 128. Accordingly, parameters of theta learner 124 may be updated to increase the misclassification loss of detector 128. In various other examples, hard negative training data may be generated. An example of hard negative training data may be where detector 128 determines that an object-of-interest is present when, in fact, no object-of-interest is represented in the image data … Generally, as described herein, hard training data and/or hard training images may refer to either hard positive or hard negative synthetic composite image data.”; “In various examples, synthetic composite image data training network 118 may be used to generate a particular number of frames of hard training images 162 … Training the detector 128 with hard training images 162 along with regular training data may improve the performance of detector 128, as the hard training images 162 represent images that the detector 128, in its previous run-time state, was unable to accurately perform object detection. Accordingly, the hard negative or hard positive training data (e.g., the hard training images or hard training data) may allow the detector 128 to reduce the number of detection “blind spots” in which the detector 128 is unable to accurately detect objects-of-interest in image data. Additionally, after training the detector using the mix of hard training images 162 and regular training data, the synthetic composite image data training network 118 may again be used to generate additional hard training images 162 for the updated detector 128. After the appropriate number of hard training images 162 are again generated, the detector 128 may be again retrained to further improve the performance of the detector 128”).
Tripathi fails to teach
wherein each evaluation image in the evaluation image dataset is divided into a grid comprising a plurality of cells; designate a cell of the grid as a misdetection region in the evaluation image dataset based on a plurality of accuracy scores corresponding to the plurality of cells; and generate a training image dataset to adjust the object detection model based on the identified misdetection region.
Tariq teaches
wherein each evaluation image in the evaluation image dataset is divided into a grid comprising a plurality of cells (Tariq, para. [0051]-[0052]: “FIG. 2A illustrates example image 100 and an example output grid 200, where each cell of the output grid 200 identifies a portion of the image 100. One example portion of the image 202 is emphasized (bolded) near the center of the image 100. It is contemplated that, as discussed herein, a “portion of the image” may include a single pixel of the image and/or a collection of pixels of the image. In some instances, an output of the machine learning model is a feature map, wherein an individual cell may represent a portion of a feature map. Such a feature map may have multiple channels, each channel associated with various element(s) determined by the model (e.g., a confidence score, a region of interest, etc.). FIG. 2A illustrates an example where a “portion of the image,” e.g., example portion 202, includes a collection of pixels of the image 100. A portion of the feature map (e.g., a cell) may be associated with a portion of the image. Example portion 202 may, therefore, be called an example cell 202 of the example output grid 200. In some instances, the example output grid 200 may be a manner of discretizing the example image 100 as output by the ML model. For example, the ML model may be configured to receive the image and output one or more ROIs and associated confidence levels per cell of the output grid 200. In at least some instances, such an output grid 200 may be discretized into m/4 by n/4 cells, according to an image of m by n pixels. In some instances, the cells may be 4 pixels by 4 pixels, though any other discretization is contemplated. In some instances, and as mentioned above, the example image 100 can be discretized into a plurality of portions of a feature map. That is, the examples are not limited to a grid of cells, and various implementations are contemplated herein.”;
PNG
media_image1.png
562
774
media_image1.png
Greyscale
PNG
media_image2.png
562
774
media_image2.png
Greyscale
);
designate a cell of the grid as a misdetection region in the evaluation image dataset based on a plurality of accuracy scores corresponding to the plurality of cells (Tariq, para. [0053]; para. [0073]-[0077]; FIG. 6A: “In some instances, an ML model may generate an ROI and/or a confidence score for each portion of the image (e.g., for each cell in FIG. 2A). For example, the ML model may determine an ROI and/or a confidence score for example portion 202 … As a non-limiting example, each cell may be associated with a center, extents, and confidence for each of a car, pedestrian, bicyclist, truck/bus, traffic light, and/or stop sign classification.”; “At operation 604, the example process 600 may include determining multiple ROIs (e.g., ROIs 400′, 404′, 408′, 412′, as illustrated in an example 612) and multiple confidence scores associated therewith, according to any of the techniques discussed herein. For example, the ML model may generate an output grid (or more generally, may output a feature map) for the image that includes output cells (or more generally, that includes a plurality of features), and may determine an ROI (and confidence score associated therewith) per classification for an output cell (e.g., each output cell) … At operation 606, the example process 600 may include receiving a reference ROI (e.g., reference ROI 500, as illustrated in an example 614), according to any of the techniques discussed herein. The reference ROI 500 (e.g., a reference region) may be ground truth received via human labeling or any other suitable method of establishing a ground truth for an area of the image that represent an object in the image. In some instances, the reference ROI may indicate an area of the image associated with a classification for which the ROIs were generated. For example, ROI 500 may indicate the area of the image representing the classification “car,” as discussed above. At operation 608, the example process 600 may include selecting a subset of examples to train the ML model, according to any of the techniques discussed herein. This may include determining a portion (e.g., an output cell) of the image, from among one or more of all the portions of the image, to include in a subset of examples for training the ML model. For example, the example process 600 may include determining a positive example (608(a)) to include in the subset, determining a negative example (608(b)) to include in the subset, and/or determining a hard example (608(c)) to include in the subset, using NMS reassignment … Operation 608(b) may include determining that a confidence score for an ROI is a minimum confidence score and/or does not meet a confidence score threshold and that a degree of alignment of the ROI to a reference ROI does not meet a threshold degree of alignment. Operation 608(a) may include selecting, as a negative example and based on this determination, the output cell and/or any of the data generated by the ML model associated therewith for inclusion in the subset for training the ML model. Turning to FIG. 6B, operation 608(c) may include (608(c)(1)) selecting a hard example, generally. Hard examples may be referred to as examples which the machine learned model gets the most wrong. For instance, such hard examples may correspond to cells having a very high confidence of a corresponding ROI, but incorrectly identify such an ROI or should otherwise be penalized based on their corresponding output. Generally, hard examples may be negative examples. Selecting a hard example may include identifying those portions of the image (e.g., one or more cells) that produced an incorrect ROI (or should otherwise be penalized), but are associated with a high confidence score; sorting the portions by confidence scores; and selecting, as a hard example and from the sorted remaining ROIs, a top number, n, of associated portions. However, it is understood that any suitable hard example selection method is contemplated.”;
PNG
media_image3.png
1156
760
media_image3.png
Greyscale
);
and generate a training image dataset to adjust the object detection model based on the identified misdetection region (Tariq, para. [0053]; para. [0073]-[0077]; see above; para. [0082]: “At operation 610, the example process 600 may include training the ML model using the selected subset of examples, according to any of the techniques discussed herein.”; see 610 in FIG. 6A above).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to: 1) modify the evaluation image dataset, as taught by Tripathi, to be divided into a grid including a plurality of cells, as taught by Tariq, 2) modify the processor that runs the evaluation image dataset on the object detection model to designate a misdetection region in the evaluation image dataset, as taught by Tripathi, to designate a cell of the grid as a misdetection region in the evaluation image dataset based on a plurality of accuracy scores corresponding to the plurality of cells, as taught by Tariq, and 3) to modify the processor that generates a training image dataset to adjust the object detection model based on the identified misdetection model, as taught by Tripathi, to generate a training image dataset to adjust the object detection model based on the identified misdetection region, wherein the identified misdetection region is a cell of the grid, as taught by Tariq.
The suggestion/motivation for doing so would have been that “it is advantageous to identify those ROIs [Region-of-interests] which the ML model got most wrong; this vastly decreases the time spent training an ML model and may increase the accuracy of the ML model since corrections made to the ML model to account for “very wrong” ROIs and/or confidence scores aren't washed out by reinforcing learning of “correct” ROIs and/or confidence scores” (Tariq, para. [0026]).
Therefore, it would have been obvious to combine Tripathi with Tariq, to obtain the invention as specified in claim 1.
Regarding claim 2, Tripathi, in view of Tariq, teaches the electronic device of claim 1, wherein the misdetection region comprises a portion of the evaluation image dataset in which the object detection model fails to accurately detect a target object (Tripathi, col. 12, lines 50-67; col. 13, lines 1-7; see rejection of claim 1 above regarding the detector trained with “hard training images” that the detector was unable to accurately detect in the previous run of the detector; examples of misdetection include images with blind spots).
Regarding claim 6, Tripathi teaches an electronic device, comprising: memory to store a background image; and a processor to: (Tripathi, col. 11, lines 51-64: “In various examples, computing device(s) 102 may be effective to implement synthetic composite image data training network 118. In various examples, computing device(s) 102 may be configured in communication such as over a network 104. Network 104 may be a wide area network, such as the internet, a local area network, and/or some combination thereof. Additionally, in various examples, computing device(s) 102 may be configured in communication with a non-transitory, computer-readable memory 103. Non-transitory, computer-readable memory 103 may be effective to store one or more instructions that, when executed by at least one processor of computing device(s) 102 program the at least one processor to perform the various techniques described herein.”)
run the evaluation image dataset on an object detection model to identify a misdetection region in the evaluation image dataset; and generate a training image dataset to adjust the object detection model based on the misdetection region (Tripathi, col. 9, lines 63-67; col. 10, lines 1-32; col. 12, lines 50-67; col. 13, lines 1-30: “As discussed in further detail below, parameters of theta learner 124 (e.g., weights and/or biases) may be updated during processing using back propagation to improve the performance of theta learner 124. The training objective of theta learner 124 may be to maximize misclassification loss 132 of detector 128. Stated another way, the training objective of theta learner 124 is to minimize the confidence score of object-of-interest detections made by detector 128 and thereby to generate “hard positive” synthetic composite image data. Hard positive synthetic composite image data is image data that includes 3D model 150 (as transformed by theta learner 124) composited with background image data 152 in which detector 128 is unable to detect any objects-of-interest—despite the presence of objects-of-interest represented within the synthetic composite image data. Misclassification loss may be loss related to object detection by detector 128. A training objective of theta learner 124 may be to increase misclassification loss of detector 128. Accordingly, parameters of theta learner 124 may be updated to increase the misclassification loss of detector 128. In various other examples, hard negative training data may be generated. An example of hard negative training data may be where detector 128 determines that an object-of-interest is present when, in fact, no object-of-interest is represented in the image data … Generally, as described herein, hard training data and/or hard training images may refer to either hard positive or hard negative synthetic composite image data.”; “In various examples, synthetic composite image data training network 118 may be used to generate a particular number of frames of hard training images 162 … Training the detector 128 with hard training images 162 along with regular training data may improve the performance of detector 128, as the hard training images 162 represent images that the detector 128, in its previous run-time state, was unable to accurately perform object detection. Accordingly, the hard negative or hard positive training data (e.g., the hard training images or hard training data) may allow the detector 128 to reduce the number of detection “blind spots” in which the detector 128 is unable to accurately detect objects-of-interest in image data. Additionally, after training the detector using the mix of hard training images 162 and regular training data, the synthetic composite image data training network 118 may again be used to generate additional hard training images 162 for the updated detector 128. After the appropriate number of hard training images 162 are again generated, the detector 128 may be again retrained to further improve the performance of the detector 128”).
Tripathi fails to teach
determine a grid of cells to divide the background image; and generate an evaluation image dataset based on a placement of a target object within the grid of cells.
Tariq teaches
determine a grid of cells to divide the background image; generate an evaluation image dataset based on a placement of a target object within the grid of cells Tariq, para. [0051]-[0052]; FIG. 2A-2B: “FIG. 2A illustrates example image 100 and an example output grid 200, where each cell of the output grid 200 identifies a portion of the image 100. One example portion of the image 202 is emphasized (bolded) near the center of the image 100. It is contemplated that, as discussed herein, a “portion of the image” may include a single pixel of the image and/or a collection of pixels of the image. In some instances, an output of the machine learning model is a feature map, wherein an individual cell may represent a portion of a feature map. Such a feature map may have multiple channels, each channel associated with various element(s) determined by the model (e.g., a confidence score, a region of interest, etc.). FIG. 2A illustrates an example where a “portion of the image,” e.g., example portion 202, includes a collection of pixels of the image 100. A portion of the feature map (e.g., a cell) may be associated with a portion of the image. Example portion 202 may, therefore, be called an example cell 202 of the example output grid 200. In some instances, the example output grid 200 may be a manner of discretizing the example image 100 as output by the ML model. For example, the ML model may be configured to receive the image and output one or more ROIs and associated confidence levels per cell of the output grid 200. In at least some instances, such an output grid 200 may be discretized into m/4 by n/4 cells, according to an image of m by n pixels. In some instances, the cells may be 4 pixels by 4 pixels, though any other discretization is contemplated. In some instances, and as mentioned above, the example image 100 can be discretized into a plurality of portions of a feature map. That is, the examples are not limited to a grid of cells, and various implementations are contemplated herein.”;
PNG
media_image1.png
562
774
media_image1.png
Greyscale
PNG
media_image2.png
562
774
media_image2.png
Greyscale
); and
designate a cell of the grid of cells as a misdetection region in the evaluation image dataset based on an accuracy score of each cell in the grid of cells (Tariq, para. [0053]; para. [0073]-[0077]; FIG. 6A: “In some instances, an ML model may generate an ROI and/or a confidence score for each portion of the image (e.g., for each cell in FIG. 2A). For example, the ML model may determine an ROI and/or a confidence score for example portion 202 … As a non-limiting example, each cell may be associated with a center, extents, and confidence for each of a car, pedestrian, bicyclist, truck/bus, traffic light, and/or stop sign classification.”; “At operation 604, the example process 600 may include determining multiple ROIs (e.g., ROIs 400′, 404′, 408′, 412′, as illustrated in an example 612) and multiple confidence scores associated therewith, according to any of the techniques discussed herein. For example, the ML model may generate an output grid (or more generally, may output a feature map) for the image that includes output cells (or more generally, that includes a plurality of features), and may determine an ROI (and confidence score associated therewith) per classification for an output cell (e.g., each output cell) … At operation 606, the example process 600 may include receiving a reference ROI (e.g., reference ROI 500, as illustrated in an example 614), according to any of the techniques discussed herein. The reference ROI 500 (e.g., a reference region) may be ground truth received via human labeling or any other suitable method of establishing a ground truth for an area of the image that represent an object in the image. In some instances, the reference ROI may indicate an area of the image associated with a classification for which the ROIs were generated. For example, ROI 500 may indicate the area of the image representing the classification “car,” as discussed above. At operation 608, the example process 600 may include selecting a subset of examples to train the ML model, according to any of the techniques discussed herein. This may include determining a portion (e.g., an output cell) of the image, from among one or more of all the portions of the image, to include in a subset of examples for training the ML model. For example, the example process 600 may include determining a positive example (608(a)) to include in the subset, determining a negative example (608(b)) to include in the subset, and/or determining a hard example (608(c)) to include in the subset, using NMS reassignment … Operation 608(b) may include determining that a confidence score for an ROI is a minimum confidence score and/or does not meet a confidence score threshold and that a degree of alignment of the ROI to a reference ROI does not meet a threshold degree of alignment. Operation 608(a) may include selecting, as a negative example and based on this determination, the output cell and/or any of the data generated by the ML model associated therewith for inclusion in the subset for training the ML model. Turning to FIG. 6B, operation 608(c) may include (608(c)(1)) selecting a hard example, generally. Hard examples may be referred to as examples which the machine learned model gets the most wrong. For instance, such hard examples may correspond to cells having a very high confidence of a corresponding ROI, but incorrectly identify such an ROI or should otherwise be penalized based on their corresponding output. Generally, hard examples may be negative examples. Selecting a hard example may include identifying those portions of the image (e.g., one or more cells) that produced an incorrect ROI (or should otherwise be penalized), but are associated with a high confidence score; sorting the portions by confidence scores; and selecting, as a hard example and from the sorted remaining ROIs, a top number, n, of associated portions. However, it is understood that any suitable hard example selection method is contemplated.”).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to: 1) modify the processor, as taught by Tripathi, to determine a grid of cells to divide the background image and generate an evaluation image dataset based on the placement of a target object within the grid of cells, as taught by Tariq, 2) modify the processor that runs the evaluation image dataset on an object detection model, as taught by Tripathi, to designate a cell of the grid of cells as a misdetection region in the evaluation image dataset based on an accuracy score of each cell in the grid of cell, as taught by Tariq.
The suggestion/motivation for doing so would have been that “it is advantageous to identify those ROIs [Region-of-interests] which the ML model got most wrong; this vastly decreases the time spent training an ML model and may increase the accuracy of the ML model since corrections made to the ML model to account for “very wrong” ROIs and/or confidence scores aren't washed out by reinforcing learning of “correct” ROIs and/or confidence scores” (Tariq, para. [0026]).
Therefore, it would have been obvious to combine Tripathi, with Tariq, to obtain the invention as specified in claim 6.
Regarding claim 9, Tripathi, in view of Tariq, teaches the electronic device of claim 6, wherein the processor is to: determine that the background image contains an object similar to the target object; and select a replacement background in response to determining that the background image contains an object similar to the target object (Tripathi, col. 10, lines 45-67; col. 11, lines 1-5; col. 13, lines 34-67: “After generating the foreground image data (e.g., the 2D representation of the 3D model as transformed by the theta learner 124) spatial transformation layer 126 may generate the synthetic composite image data by rendering the foreground image data with the background image data 152 to generate synthetic composite image data. In various examples, the spatial transformation layer 126 may composite the foreground image data output by DR component 125 … by replacing a subset of the pixel values of background image data 152 with pixel values of the foreground image data, as transformed by the operations determined by theta learner 124. In various examples, to generate the synthetic composite image data, the spatial transformation layer 126 may evaluate pixels of the background image data 152 on a pixel-by-pixel basis to determine whether or not the pixel values at each pixel address should be replaced by a pixel value of the foreground image data. In various examples, the spatial transformation layer 126 may determine a plurality of pixel addresses in the background image data 152 at which to render the 2D representation of the transformed 3D model 150 (e.g., the transformed representation of the foreground object-of-interest)”; “In various examples, synthetic composite image data training network 118 may be used to generate a particular number of hard training images 162. For example, a target number and/or threshold number of composite hard training images 162 may be determined. After the target and/or threshold number is reached, the hard training images 162 may be mixed with training data of training dataset 202 (e.g., a default training dataset for detector 128). Hard training images 162 may be frames of background image data 152 where pixel values of particular pixels of the background image have been replaced by pixel values of the segmented foreground image of the 3D model 150, as transformed by theta learner 124, DR component 125, and/or spatial transformation layer 126.”).
Regarding claim 10, Tripathi, in view of Tariq, teaches the electronic device of claim 9, wherein the processor is to determine that the background image contains an object similar to the target object based on metadata of the background image and a target object image (Tripathi, col. 10, lines 45-67; col. 11, lines 1-5; col. 13, lines 34-67; see rejection of claim 9 above; the pixel values of the background image is example of metadata).
Regarding claim 11, Tripathi teaches a non-transitory tangible computer-readable medium comprising instructions when executed cause a processor of an electronic device to: (Tripathi, col. 11, lines 50-64: “additionally, in various examples, computing device(s) 102 may be configured in communication with a non-transitory, computer-readable memory 103. Non-transitory, computer-readable memory 103 may be effective to store one or more instructions that, when executed by at least one processor of computing device(s) 102 program the at least one processor to perform the various techniques described herein.”)
find a blind spot in an object detection model; generate a training image dataset having a target object positioned in the blind spot; and update the object detection model by learning using the training image dataset (Tripathi, col. 12, lines 50-67; col. 13, lines 1-30: “In various examples, synthetic composite image data training network 118 may be used to generate a particular number of frames of hard training images 162 … Training the detector 128 with hard training images 162 along with regular training data may improve the performance of detector 128, as the hard training images 162 represent images that the detector 128, in its previous run-time state, was unable to accurately perform object detection. Accordingly, the hard negative or hard positive training data (e.g., the hard training images or hard training data) may allow the detector 128 to reduce the number of detection “blind spots” in which the detector 128 is unable to accurately detect objects-of-interest in image data. Additionally, after training the detector using the mix of hard training images 162 and regular training data, the synthetic composite image data training network 118 may again be used to generate additional hard training images 162 for the updated detector 128. After the appropriate number of hard training images 162 are again generated, the detector 128 may be again retrained to further improve the performance of the detector 128”).
Tripathi fails to teach
find a blind spot in an object detection model based on an accuracy score of each cell in a plurality of cells, wherein the blind spot corresponds to a cell of an evaluation image dataset; and generate a training image dataset having a target object positioned in the cell of the blind spot.
Tariq teaches
find a blind spot in an object detection model based on an accuracy score of each cell in a plurality of cells, wherein the blind spot corresponds to a cell of an evaluation image dataset (Tariq, para. [0053]; para. [0073]-[0077]; FIG. 6A: “In some instances, an ML model may generate an ROI and/or a confidence score for each portion of the image (e.g., for each cell in FIG. 2A). For example, the ML model may determine an ROI and/or a confidence score for example portion 202 … As a non-limiting example, each cell may be associated with a center, extents, and confidence for each of a car, pedestrian, bicyclist, truck/bus, traffic light, and/or stop sign classification.”; “At operation 604, the example process 600 may include determining multiple ROIs (e.g., ROIs 400′, 404′, 408′, 412′, as illustrated in an example 612) and multiple confidence scores associated therewith, according to any of the techniques discussed herein. For example, the ML model may generate an output grid (or more generally, may output a feature map) for the image that includes output cells (or more generally, that includes a plurality of features), and may determine an ROI (and confidence score associated therewith) per classification for an output cell (e.g., each output cell) … At operation 606, the example process 600 may include receiving a reference ROI (e.g., reference ROI 500, as illustrated in an example 614), according to any of the techniques discussed herein. The reference ROI 500 (e.g., a reference region) may be ground truth received via human labeling or any other suitable method of establishing a ground truth for an area of the image that represent an object in the image. In some instances, the reference ROI may indicate an area of the image associated with a classification for which the ROIs were generated. For example, ROI 500 may indicate the area of the image representing the classification “car,” as discussed above. At operation 608, the example process 600 may include selecting a subset of examples to train the ML model, according to any of the techniques discussed herein. This may include determining a portion (e.g., an output cell) of the image, from among one or more of all the portions of the image, to include in a subset of examples for training the ML model. For example, the example process 600 may include determining a positive example (608(a)) to include in the subset, determining a negative example (608(b)) to include in the subset, and/or determining a hard example (608(c)) to include in the subset, using NMS reassignment … Operation 608(b) may include determining that a confidence score for an ROI is a minimum confidence score and/or does not meet a confidence score threshold and that a degree of alignment of the ROI to a reference ROI does not meet a threshold degree of alignment. Operation 608(a) may include selecting, as a negative example and based on this determination, the output cell and/or any of the data generated by the ML model associated therewith for inclusion in the subset for training the ML model. Turning to FIG. 6B, operation 608(c) may include (608(c)(1)) selecting a hard example, generally. Hard examples may be referred to as examples which the machine learned model gets the most wrong. For instance, such hard examples may correspond to cells having a very high confidence of a corresponding ROI, but incorrectly identify such an ROI or should otherwise be penalized based on their corresponding output. Generally, hard examples may be negative examples. Selecting a hard example may include identifying those portions of the image (e.g., one or more cells) that produced an incorrect ROI (or should otherwise be penalized), but are associated with a high confidence score; sorting the portions by confidence scores; and selecting, as a hard example and from the sorted remaining ROIs, a top number, n, of associated portions. However, it is understood that any suitable hard example selection method is contemplated.”;
PNG
media_image3.png
1156
760
media_image3.png
Greyscale
); and
generate a training image dataset having a target object positioned in the cell of the blind spot (Tariq, para. [0053]; para. [0073]-[0077]; see above; para. [0082]: “At operation 610, the example process 600 may include training the ML model using the selected subset of examples, according to any of the techniques discussed herein.”; see 610 in FIG. 6A above).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to: 1) modify the instructions including finding a blind spot in an object detection model, as taught by Tripathi, to be based on an accuracy score of each cell in a plurality of cells, wherein the blind spots corresponds to a cell of an evaluation image dataset, as taught by Tariq, 2) modify the instructions including generating a training image dataset having a target object positioned in blind spot, as taught by Tripathi, to include having a target object positioned in the cell of the blind spot, as taught by Tariq.
The suggestion/motivation for doing so would have been that “it is advantageous to identify those ROIs [Region-of-interests] which the ML model got most wrong; this vastly decreases the time spent training an ML model and may increase the accuracy of the ML model since corrections made to the ML model to account for “very wrong” ROIs and/or confidence scores aren't washed out by reinforcing learning of “correct” ROIs and/or confidence scores” (Tariq, para. [0026]).
Therefore, it would have been obvious to combine Tripathi, with Tariq, to obtain the invention a as specified in claim 11.
Regarding claim 12, Tripathi, in view of Tariq, teaches the non-transitory tangible computer-readable medium of claim 11, wherein the instructions to generate the training image dataset comprise instructions that when executed cause the processor to: determine placement of the target object in the training image dataset based on a misdetection region in an evaluation image dataset (Tripathi, col. 12, lines 50-67; col. 13, lines 1-30; see rejection of claim 11 above).
Regarding claim 13, Tripathi, in view of Tariq, teaches the non-transitory tangible computer-readable medium of claim 11, wherein the instructions when executed cause the processor to: determine a grid used to generate the evaluation image dataset; select a background image for the training image dataset; and combine the target object and the background image to generate an image for the training image dataset (Tripathi, col. 12, lines 50-67; col. 13, lines 1-30; see rejection of claim 11 above; Tripathi, col. 4, lines 54-67; col. 5, lines 1-35: “After determining the various operations to be performed on the 3D model, a differentiable renderer component (e.g., a neural mesh renderer (NMR)) may be used to determine a 2D representation of the 3D model, as transformed according to the operations determined by the theta learner. The differentiable rendering component may be referred to herein as a DR component … For brevity, the 2D representation of the 3D model generated by the DR component (as transformed by the theta learner) may be referred to herein as “foreground image data.” This terminology may be used even in the case where the 2D representation of the 3D model is combined with the background image data as part of a composite image … After determining the foreground image data (as transformed by the theta learner), a spatial transformation layer may combine the transformed 2D representation of the 3D model (e.g., the foreground image data) with the background image data to generate a frame of synthetic composite image data (sometimes referred to herein as a frame of composite image data). The spatial transformation layer may perform the operations determined by the theta learner on the foreground image data in order to combine the foreground image data with the background image data. The spatial transformation layer may generate the synthetic composite image data by compositing the background image data and the foreground image data (as transformed according to the operations determined by the theta learner and as represented in two dimensions by the DR component) on a pixel-by-pixel basis … For example, a set of pixels representing a human may be composited in a background image, by replacing pixel values of the background image with pixel values representing the human.”; “Once a sufficient number (e.g., a threshold number) of hard positives have been generated, the detector may be retrained, by mixing the hard positive image data with other training data (e.g., with real/natural annotated images). Hard positive image data may be especially beneficial for training the detector, as the hard positive image data represents “blind spots” of the training data distribution used to train the current iteration of the detector.”).
Regarding claim 14, Tripathi, in view of Tariq, teaches the non-transitory tangible computer-readable medium of the non-transitory tangible computer-readable medium of wherein the instructions when executed cause the processor to: record a position of the target object in the training image dataset as a ground truth bounding box (Tripathi, col. 11, lines 33-50: “The synthetic composite image data generated by spatial transformation layer 126 may be sent to detector 128. Detector 128 may be a single shot detector (SSD) or other detector effective to locate and/or classify objects-of-interest in image data. For example, detector 128 may be an object detector that has been trained to detect dogs. Detector 128 may generate a bounding box that identifies the location of a dog detected in the input image data (e.g., in the synthetic composite image data generated by synthetic composite image data training network 118). Misclassification loss 132 may occur when detector 128 misidentifies a dog. Stated another way, misclassification loss 132 may occur when detector 128 identifies a dog in image data where no dog is present and/or when detector 128 identifies a non-dog object as a dog. In various examples, for each bounding box generated by detector 128, detector 128 may generate a confidence score indicating a confidence that detector 128 has correctly identified a dog bounded by the bounding box.”).
Regarding claim 15, Tripathi, in view of Tariq, teaches the non-transitory tangible computer-readable medium of claim 11, wherein the instructions when executed cause the processor to: perform the transfer learning to train the object detection model with the training image dataset (Tripathi, col. 12, lines 50-67; col. 13, lines 1-30; see rejection of claim above; Transfer learning is a machine learning technique where knowledge gained from solving one task is applied to improve the performance on a related, but different, task; this is shown in the process of creating frames of synthetic composite image data by combining the first foreground image data with the first background image data and then training a machine learning model using the hard negative training data to minimize error in recognizing specific human objects; Tripathi, col. 7, lines 59-67: “In various examples, the 3D model may be of an object class that corresponds to an object class for which detector 128 is designed to detect. For example, if detector 128 is designed to detect cars, 3D model(s) 150 may include various 3D models of different types of cars.”; therefore, by creating new synthetic training data for training the machine learning model, transfer learning is happening by changing the generic object detection model to a model specifically trained for more detailed/specific/accurate detection of cars).
Regarding claim 15, Tripathi, in view of Tariq, teaches the non-transitory tangible computer-readable medium of claim 11, wherein the instructions where executed cause the processor to: perform the learning to train the object detection model with the training image dataset (Tariq, para. [0082]; see rejection of claim 11 above).
Regarding claim 16, Tripathi, in view of Tariq, teaches the electronic device of claim 1, wherein an accuracy score corresponding to the designation cells indicates that the detection model failed to detect the target object within that cell (Tariq, para. [0053]; para. [0073]-[0077]; FIG. 6A; see rejection of claim 1 above discussing negative and hard examples found in cells of the grid that fail to identify the object correctly).
Regarding claim 17, Tripathi, in view of Tariq, teaches the electronic device of claim 6, wherein the misdetection region is further designated based on a comparison of the accuracy score and a threshold accuracy score (Tariq, para. [0053]; para. [0073]-[0077]; FIG. 6A; see rejection of claim 1 above discussing the receive a reference ROI 606 step that compares the objects identified in each cell with the reference and deciding if the cell indicates a positive example, negative example, or a hard example based off the confidence threshold comparison of the ROI reference and the confidence score of each cell in the grid).
Regarding claim 18, Tripathi, in view of Tariq, teaches the non-transitory tangible computer-readable medium of claim 11.
Tripathi, in view of Tariq, fails to teach
wherein the accuracy score is calculated based on a number of times the object detection model accurately detects a target object located in each cell.
Tariq further teaches
wherein the accuracy score is calculated based on a number of times the object detection model accurately detects a target object located in each cell (Tariq, para. [0027]: “In some instances, the techniques discussed herein may include selecting particular examples for training the ML model. Selecting these examples may include hard example mining, for example, which may include sorting multiple ROIs by confidence scores (e.g., greatest confidence scores to least) and/or error in confidence score (e.g., a confidence score error associated with an ROI, for an ROI that was suppressed according to NMS) and choosing the top n number of ROIs. In some instances, selecting examples by hard example mining may exclude the ROI associated with a maximum confidence score (or scores). Additionally, or alternatively, the techniques may include choosing n number of random ROIs. In some instances, the number, n, may be chosen to be the number of positive examples in the image (e.g., positively identified ROIs corresponding to objects represented in the image).”).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the accuracy score, as taught by Tripathi, in view of Tariq, to be calculated based on a number of times the object detection model accurately detects a target object located in each cell, as further taught by Tariq.
The suggestion/motivation for doing so would have been to decrease the likelihood of falsely identifying false positives in object detection (i.e. recognize an object in the image even though it is not there).
Therefore, it would have been obvious to combine Tripathi and Tariq, with Tariq further, to obtain the invention as specified in claim 18.
Claims 3-5 and 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Tripathi, in view of Tariq, and in view of U.S. Patent Application Publication No.: 2019/0340785 (Lopich et al.) (hereinafter Lopich).
Regarding claim 3, Tripathi, in view of Tariq, teaches the electronic device of claim 1, wherein the processor to generate the evaluation image dataset comprises the processor to: divide a background image into a grid (Tariq, para. [0051]-[0052]; FIG. 2; see rejection of claim 1 above).
Tripathi, in view of Tariq, fails to teach
place a target object into a cell of the grid.
Lopich teaches
place a target object into a cell of the grid (Lopich, para. [0046]-[0047]; FIG. 2: “In some examples, the characteristic of the object detection data comprises an indication that an object, detected in at least one of the first and second frames, has been detected in a predetermined portion of the respective image frame. In some cases, the characteristic comprises an indication that the detected object has been detected as being located in a predetermined region of the environment represented in the respective image frame. FIG. 2 shows schematically an example of generating HOG features as part of the feature extraction operation. An input image frame 200, e.g. the first or second image frame described above, is divided into a grid of portions (or “cells”) 210. This grid may be predefined, such that each frame can be said to be divided into the same grid. For example, the cells 210 may have dimensions of 7×7 elements (or “pixels”) 220 as shown in FIG. 2. In other examples, the cells may have different dimensions of pixels 220.”);
PNG
media_image4.png
573
479
media_image4.png
Greyscale
).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the processor, generating the evaluation dataset, as taught by Tripathi, in view of Tariq, to place a target object into a cell of the grid, as taught by Lopich.
The suggestion/motivation for doing so would have been that placing an object onto a background grid for image processing provides several key advantages, primarily by simplifying the scene, enhancing accuracy, and enabling automated, high-speed measurements; grids serve as known, consistent references that help algorithms distinguish foreground from background and calculate scale.
Therefore, it would have been obvious to combine Tripathi and Tariq, with Lopich, to obtain the invention as specified in claim 3.
Regarding claim 4, Tripathi, in view of Tariq, and in view of Lopich, teaches the electronic device of claim 3, wherein the evaluation image dataset comprises a first image with the target object placed in a first cell of the grid and a second image with the target object placed in a second cell of the grid (Lopich, para. [0046]-[0047]; FIG. 2; see rejection of claim 3 above; each cell grid contains an object using Histogram-of-Objects).
Regarding claim 5, Tripathi, in view of Tariq, teaches the electronic device of claim 1.
Tripathi, in view of Tariq, fails to teach
wherein the processor to generate the evaluation image dataset further comprises the processor to size a target object to fit within a cell of a grid.
Lopich teaches
wherein the processor to generate the evaluation image dataset further comprises the processor to size a target object to fit within a cell of a grid (Lopich, para. [0044]; para. [0054]: “In some examples, the characteristic of the object detection data comprises an indication that a change in the position or size of the object detected in the first and second image frames, relative to the environment, has a predetermined relationship with a predetermined threshold. For example, the object detection data may indicate that the object has changed in its position relative to the environment, e.g. a fixed coordinate system, by more than the predetermined threshold. The predetermined relationship between the positional change of the object and the predetermined threshold may comprise the positional change of the object being less than, less than or equal to, equal to, greater than or equal to, or greater than the predetermined threshold.”; “In examples, the at least one modified feature extraction parameter comprises a relative scale (or “scale ratio”) between the given image frame 200 and the dimensions of the cells 210 (or “cell size”). For example, in response to the characteristic of the object detection data, the cell size and/or the frame size may be modified when processing the one or more further image frames. The image frame may be rescaled, e.g. downscaled, by a scale factor such that a given cell size has a different area coverage of the rescaled image frame. The cell size may additionally or alternatively be modified to give the same effect.”).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the processor, generating the evaluation dataset, as taught by Tripathi, in view of Tariq, to size a target object to fit within a cell of a grid, as taught by Lopich.
The suggestion/motivation for doing so would have been to more accurately capture the entirety of an object so it does not go out of bounds of the grid of an image frame, which will improve object detection algorithms.
Therefore, it would have been obvious to combine Tripathi and Tariq, with Lopich, to obtain the invention as specified in claim 5.
Regarding claim 7, Tripathi, in view of Tariq, teaches the electronic device of claim 6.
Tripathi, in view of Tariq, fails to teach
wherein the processor is to: load the target object from an image database; determine a size of the target object to be placed in a cell of the background image; and adjust the target object to the determined size.
Lopich teaches
wherein the processor is to: load the target object from an image database; determine a size of the target object to be placed in a cell of the background image; and adjust the target object to the determined size (Lopich, para. [0044]; para. [0054]: “In some examples, the characteristic of the object detection data comprises an indication that a change in the position or size of the object detected in the first and second image frames, relative to the environment, has a predetermined relationship with a predetermined threshold. For example, the object detection data may indicate that the object has changed in its position relative to the environment, e.g. a fixed coordinate system, by more than the predetermined threshold. The predetermined relationship between the positional change of the object and the predetermined threshold may comprise the positional change of the object being less than, less than or equal to, equal to, greater than or equal to, or greater than the predetermined threshold.”; “In examples, the at least one modified feature extraction parameter comprises a relative scale (or “scale ratio”) between the given image frame 200 and the dimensions of the cells 210 (or “cell size”). For example, in response to the characteristic of the object detection data, the cell size and/or the frame size may be modified when processing the one or more further image frames. The image frame may be rescaled, e.g. downscaled, by a scale factor such that a given cell size has a different area coverage of the rescaled image frame. The cell size may additionally or alternatively be modified to give the same effect.”).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the processor, as taught by Tripathi, in view of Tariq, to load the target object from an image database; determine a size of the target object to be placed in a cell of the background image; and adjust the target object to the determined size, as taught by Lopich.
The suggestion/motivation for doing so would have been to more accurately capture the entirety of an object so it does not go out of bounds of the grid of an image frame, which will improve object detection algorithms; this prevents partial capturing of objects in images.
Therefore, it would have been obvious to combine Tripathi and Lopich, with Lopich, to obtain the invention as specified in claim 7.
Regarding claim 8, Tripathi, in view of Tariq, teaches the electronic device of claim 6.
Tripathi, in view of Tariq, fails to teach
wherein the processor to generate the evaluation image dataset comprises the processor to: place the target object in a first cell of the background image; place the target object in a second cell of the background image; render a first image for the first target object placement; and render a second image for the second target object placement.
Lopich teaches
wherein the processor to generate the evaluation image dataset comprises the processor to: place the target object in a first cell of the background image; place the target object in a second cell of the background image; render a first image for the first target object placement; and render a second image for the second target object placement (Lopich, para. [0046]-[0047]; FIG. 2: “In some examples, the characteristic of the object detection data comprises an indication that an object, detected in at least one of the first and second frames, has been detected in a predetermined portion of the respective image frame. In some cases, the characteristic comprises an indication that the detected object has been detected as being located in a predetermined region of the environment represented in the respective image frame. FIG. 2 shows schematically an example of generating HOG features as part of the feature extraction operation. An input image frame 200, e.g. the first or second image frame described above, is divided into a grid of portions (or “cells”) 210. This grid may be predefined, such that each frame can be said to be divided into the same grid. For example, the cells 210 may have dimensions of 7×7 elements (or “pixels”) 220 as shown in FIG. 2. In other examples, the cells may have different dimensions of pixels 220.”);
PNG
media_image4.png
573
479
media_image4.png
Greyscale
).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify the processor, as taught by Tripathi, in view of Tariq, to place the target object in a first cell of the background image; place the target object in a second cell of the background image; render a first image for the first target object placement; and render a second image for the second target object placement, as taught by Lopich.
The suggestion/motivation for doing so would have been that placing an object onto a background grid for image processing provides several key advantages, primarily by simplifying the scene, enhancing accuracy, and enabling automated, high-speed measurements; grids serve as known, consistent references that help algorithms distinguish foreground from background and calculate scale.
Therefore, it would have been obvious to combine Tripathi and Tariq, with Lopich, to obtain the invention as specified in claim 8.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: U.S. Patent Application Publication No.: 2024/0029394 and 2022/0101047.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL ADAM SHARIFF whose telephone number is 571-272-9741. The examiner can normally be reached M-F 8:30-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached on 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ADAM SHARIFF/
Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672