DETAILED ACTION
This office action is responsive to application 18/923,330 filed on October 22, 2024. Claims 1-20 are pending in the application and have been examined by the Examiner.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 2, 6-9, 12, 14, 17, 18 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kim et al. (US 2021/0124909).
Consider claim 1, Kim et al. teaches:
A camera (see figures 1A and 2), comprising:
an image capture device (camera, 110, paragraphs 0031 and 0032) to output one or more images (object image, 310, figure 6, paragraph 0096); and
one or more processors (processor, 140, paragraph 0044) coupled with memory (volatile memory, 130, paragraphs 0034 and 0045), the one or more processors (140) to:
detect a feature represented in a first image of the one or more images (For instance, the processor (140) detects an avoidance or non-avoidance feature via an avoidance identification model (500), figure 6, paragraphs 0096 and 0099.);
select, based on the feature, a machine vision model to process at least one of the first image or a second image of the one or more images (For instance, object type identification model 600 or object type identification model 610 is selected based on whether the feature is an avoidance feature or a non-avoidance feature, figure 6, paragraphs 0099 and 0100.);
allocate, to the machine vision model (600 or 610), based on a characteristic of the machine vision model, a portion of the memory (For instance, a “2 Level” portion of the memory (130) is allocated to the machine vision model (600 or 610), figures 6 and 1A, paragraphs 0099 and 0049.); and
execute the machine vision model (600 or 610) using the portion of the memory (130) to process the at least one of the first image or the second image (i.e. to identify a type of object in the image using the machine vision model (600 or 610), paragraphs 0100 and 0101, see figure 6).
Consider claim 2, and as applied to claim 1 above, Kim et al. further teaches that the machine vision model (600 or 610, figure 6) is a second machine vision model (i.e. after the avoidance identification model, 500, figure 6), the portion of the memory (130) is a second portion (“2 Level”, see figures 1A and 6) of the memory (130), and the one or more processors (140) are to: detect the feature (i.e. the avoidance or non-avoidance feature) represented in the first image (310) by causing a first machine vision model (500) to execute on a first portion (“1 Level”, see figures 1A and 6) of memory (130, see paragraphs 0095-0101).
Consider claim 6, and as applied to claim 1 above, Kim et al. further teaches that the portion of the memory is a second portion of the memory (i.e. “2 Level” of the volatile memory (130), see claim 1 rationale), and one or more processors (140) are to: retrieve a data structure (i.e. the object model) corresponding to the machine vision model from a first portion of the memory (i.e. from a non-volatile memory (120) of the memory, paragraph 0049); and deploy the machine vision model (600, 610) on the second portion of the memory (130) using the retrieved data structure (see paragraphs 0099 and 0049).
Consider claim 7, and as applied to claim 1 above, Kim et al. further teaches that the one or more processors (140) are to select the machine vision model from a plurality of machine vision models (e.g. from 600 or 610, paragraphs 0099 and 0100).
Consider claim 8, and as applied to claim 1 above, Kim et al. further teaches that the machine vision model (600 or 610) comprises a neural network (“neural network”, paragraph 0060) to perform at least one of object detection (i.e. to identify a type of object in the image using the machine vision model (600 or 610), paragraphs 0100 and 0101, see figure 6).
Consider claim 9, and as applied to claim 1 above, Kim et al. further teaches that the one or more processors (140) are to allocate the portion of memory (130) to include at least one of a minimum threshold amount of the memory or a maximum threshold amount of the memory (i.e. the minimum or maximum threshold amount of memory necessary to hold the machine vision model (600 or 610), figures 6 and 1A, paragraph 0099).
Consider claim 12, Kim et al. teaches:
A method, comprising:
selecting, by processing circuitry (processor, 140, paragraph 0044) of a camera (see figures 1A and 2), a machine vision model (600 or 610, figure 6) to process one or more images (object image, 310, figure 6, paragraph 0096) from an image capture device (camera, 110, paragraphs 0031 and 0032) of the camera (For instance, object type identification model 600 or object type identification model 610 is selected based on whether the feature is an avoidance feature or a non-avoidance feature, figure 6, paragraphs 0099 and 0100.);
allocating a portion of memory (volatile memory, 130, paragraphs 0034 and 0045) of the camera (see figures 1A and 2), by the processing circuitry (140), based on a characteristic of at least one of the one or more images or the machine vision model (For instance, a “2 Level” portion of the memory (130) is allocated to the machine vision model (600 or 610), figures 6 and 1A, paragraphs 0099 and 0049.); and
executing, by the processing circuitry (140), the machine vision model (600 or 610) on the portion of the memory (130) to process the one or more images (i.e. to identify a type of object in the image using the machine vision model (600 or 610), paragraphs 0100 and 0101, see figure 6).
Consider claim 14, and as applied to claim 12 above, Kim et al. further teaches that the machine vision model (600 or 610, figure 6) is a second machine vision model (i.e. after the avoidance identification model, 500, figure 6), the portion of the memory (130) is a second portion (“2 Level”, see figures 1A and 6) of the memory (130), and the one or more processors (140) are to: detect the feature (i.e. the avoidance or non-avoidance feature) represented in the first image (310) by causing a first machine vision model (500) to execute on a first portion (“1 Level”, see figures 1A and 6) of memory (130, see paragraphs 0095-0101).
Consider claim 17, and as applied to claim 12 above, Kim et al. further teaches that the machine vision model (600 or 610) comprises a second machine vision model (i.e. after the avoidance identification model, 500, figure 6), the method further comprising: detecting, by the processing circuitry (140), using a first machine vision model (500), a feature of an object represented in the one or more images (For instance, the processor (140) detects an avoidance or non-avoidance feature via an avoidance identification model (500), figure 6, paragraphs 0096 and 0099.); selecting, by the processing circuitry (140), the second machine vision model (600 or 610) based on the feature (For instance, object type identification model 600 or object type identification model 610 is selected based on whether the feature is an avoidance feature or a non-avoidance feature, figure 6, paragraphs 0099 and 0100.); and processing, by the second machine vision model (600 or 610), the feature to detect a characteristic of the feature (i.e. to identify a type of object in the image using the machine vision model (600 or 610), paragraphs 0100 and 0101, see figure 6).
Consider claim 18, Kim et al. teaches:
A system (see figures 1A and 2), comprising:
one or more processors (processor, 140, paragraph 0044) to:
detect an object in image data (object image, 310, figure 6, paragraph 0096) from an image capture device (camera, 110, paragraphs 0031 and 0032), the object having a class (For instance, the processor (140) detects an avoidance or non-avoidance object (avoidance object or non-avoidance object are classes) via an avoidance identification model (500), figure 6, paragraphs 0096 and 0099.);
select, based on the class, a machine vision model to process the image stream (For instance, object type identification model 600 or object type identification model 610 is selected based on whether the class is an avoidance object or a non-avoidance object, figure 6, paragraphs 0099 and 0100.);
determine a target memory usage for the machine vision model (For instance, a “2 Level” portion of the memory (130) is determined as a target memory usage for the machine vision model (600 or 610), figures 6 and 1A, paragraphs 0099 and 0049.); and
execute the machine vision model (600 or 610), on a portion (i.e. the “2 Level” portion) of memory (130) corresponding to the target memory usage (figures 6 and 1A), to generate an output regarding the object based on the image data (i.e. to identify a type of object in the image using the machine vision model (600 or 610), paragraphs 0100 and 0101, see figure 6).
Consider claim 20, and as applied to claim 18 above, Kim et al. further teaches that the machine vision model (600 or 610) is to determine a state of the object based on the image data (i.e. to determine a specific avoidance or non-avoidance object, figure 6, paragraphs 0099-0101).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3, 11 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 2021/0124909) in view of West (US 7,893,943).
Consider claim 3, and as applied to claim 1 above, Kim et al. does not explicitly teach that the one or more processors are to determine an amount of the portion of memory to allocate to the machine vision model based on at least one of a frame rate associated with the one or more images or a resolution of the one or more images.
West similarly teaches a camera (figure 1) comprising an image capture device (video source, 112, column 2, lines 58 and 59) and a memory controller (208, figure 2) for allocating memory (column 4, lines 27-40).
However, West additionally teaches that the memory controller (208) determines an amount of the portion of memory to allocate based on a resolution of the one or more images (“the memory controller 208 dynamically allocates a buffer 210 sized proportionate to a horizontal resolution of the digital image data 202”, column 4, lines 27-40).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have a portion of the memory allocated to the machine vision model by the one or more processors taught by Kim et al. be sized based on a resolution of the one or more images as taught by West for the benefit of optimizing the memory to provide maximum memory bandwidth (West, column 4, lines 35-40).
Consider claim 11, and as applied to claim 1 above, Kim et al. does not explicitly teach that the image capture device is to output the first image and the second image as a sequence of images at a frame rate.
West similarly teaches a camera (figure 1) comprising an image capture device (video source, 112, column 2, lines 58 and 59) and a memory controller (208, figure 2) for allocating memory (column 4, lines 27-40).
However, West additionally teaches that the image capture device (112) is to output the first image and the second image (i.e. digital image data 108, figure 1) as a sequence of images at a frame rate (see “frame rate”, column 3, lines 6-13).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have image capture device taught by Kim et al. output the first image and the second image as a sequence of images at a frame rate as taught by West for the benefit of enabling higher quality images (West, column 2, lines 18-20).
Consider claim 13, and as applied to claim 12 above, Kim et al. does not explicitly teach that the characteristic comprises at least one of a frame rate associated with the one or more images or a resolution of the one or more images.
West similarly teaches a camera (figure 1) comprising an image capture device (video source, 112, column 2, lines 58 and 59) and a memory controller (208, figure 2) for allocating memory (column 4, lines 27-40).
However, West additionally teaches that the memory controller (208) determines an amount of the portion of memory to allocate based on a resolution of the one or more images (“the memory controller 208 dynamically allocates a buffer 210 sized proportionate to a horizontal resolution of the digital image data 202”, column 4, lines 27-40).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have the characteristic taught by Kim et al. be a resolution of the one or more images as taught by West for the benefit of optimizing the memory to provide maximum memory bandwidth (West, column 4, lines 35-40).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 2021/0124909) in view of Jung et al. (US 2021/0201447).
Consider claim 10, and as applied to claim 1 above, Kim et al. does not explicitly teach that the image capture device is to output the one or more images according to a predetermined quantization, and the machine vision model is configured to process the one or more images according to the predetermined quantization.
Jung et al. similarly teaches an image capture device (110, figure 1) outputting an image to an object recognition apparatus (120, paragraph 0054).
However, Jung et al. additionally teaches that the image is output and processed according to a predetermined quantization (see paragraphs 0056-0058).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have the one or more images taught by Kim et al. be output and processed according to a predetermined quantization as taught by Jung et al. for the benefit of reducing an amount of data and bandwidth (Jung et al., paragraph 0056).
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 2021/0124909) in view of Sugata et al. (US 2011/0307591).
Consider claim 4, and as applied to claim 1 above, Kim et al. does not explicitly teach that the one or more processors are to: monitor a utilization of the memory by the machine vision model; and output an alert responsive to the utilization exceeding a threshold.
Sugata et al. similarly teaches one or more processors (microprocessor, 101, figure 2) performing an image delivery program (see paragraph 0066).
However, Sugata et al. additionally teaches that the one or more processors monitor a utilization of the memory and output an alert responsive to the utilization exceeding a threshold (“an alert will be output when the memory utilization reaches either the warning threshold or the error threshold” paragraph 0114).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to monitor the memory utilization of the machine vision model taught by Kim et al. and output an alert responsive to the utilization exceeding a threshold as taught by Sugata et al. for the benefit of making it possible to more simply manage the target apparatus (Sugata et al., paragraph 0006).
Consider claim 15, and as applied to claim 12 above, Kim et al. does not explicitly teach monitoring, by the processing circuitry, a utilization of the portion of the memory by the machine vision model; and output an alert responsive to the utilization exceeding a threshold.
Sugata et al. similarly teaches one or more processors (microprocessor, 101, figure 2) performing an image delivery program (see paragraph 0066).
However, Sugata et al. additionally teaches that the one or more processors monitor a utilization of a portion of the memory and output an alert responsive to the utilization exceeding a threshold (“an alert will be output when the memory utilization reaches either the warning threshold or the error threshold” paragraph 0114).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to monitor the memory utilization of the machine vision model taught by Kim et al. and output an alert responsive to the utilization exceeding a threshold as taught by Sugata et al. for the benefit of making it possible to more simply manage the target apparatus (Sugata et al., paragraph 0006).
Claims 5 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 2021/0124909) in view of Yuan et al. (US 2024/0135180).
Consider claim 5, and as applied to claim 1 above, Kim et al. does not explicitly teach that the one or more processors are to: monitor a utilization of the memory by the machine vision model; and modify operation of the machine vision model responsive to the utilization exceeding a threshold.
Yuan et al. similarly teaches a machine vision model in the form of a Deep Neural Network (DNN) used for image classification and video recognition (see paragraph 0053).
However, Yuan et al. additionally teaches monitoring a utilization of the memory by the machine vision model, and modifying operation of the machine vision model responsive to the utilization exceeding a threshold (“In some embodiments, the resource capacity metric may define a maximum memory available for the DNN model. Thus, the dual problem training engine 120 may perform training iterations until the tensor rank is minimized to below the resource capacity metric such that the DNN model being decomposed to have one or more tensors of the tensor rank satisfies the maximum memory capacity. Accordingly, the dual problem training engine 120 may determine the minimum tensor rank associated with the available memory capacity and terminate training iterations where the minimization of the loss converges after the tensor rank is minimized to the minimum tensor rank associated with the available memory capacity.” paragraph 0087).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have the one or more processors taught by Kim et al. monitor and modify the machine vision model as taught by Yuan et al. for the benefit of satisfying a maximum memory capacity metric (Yuan et al., paragraphs 0087 and 0006).
Consider claim 19, and as applied to claim 18 above, Kim et al. does not explicitly teach that the one or more processors are to: monitor utilization of the portion of memory by the machine vision model; and update operation of the machine vision model responsive to the utilization exceeding a threshold.
Yuan et al. similarly teaches a machine vision model in the form of a Deep Neural Network (DNN) used for image classification and video recognition (see paragraph 0053).
However, Yuan et al. additionally teaches monitoring a utilization of the memory by the machine vision model, and modifying operation of the machine vision model responsive to the utilization exceeding a threshold (“In some embodiments, the resource capacity metric may define a maximum memory available for the DNN model. Thus, the dual problem training engine 120 may perform training iterations until the tensor rank is minimized to below the resource capacity metric such that the DNN model being decomposed to have one or more tensors of the tensor rank satisfies the maximum memory capacity. Accordingly, the dual problem training engine 120 may determine the minimum tensor rank associated with the available memory capacity and terminate training iterations where the minimization of the loss converges after the tensor rank is minimized to the minimum tensor rank associated with the available memory capacity.” paragraph 0087).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have the one or more processors taught by Kim et al. monitor and modify the machine vision model as taught by Yuan et al. for the benefit of satisfying a maximum memory capacity metric (Yuan et al., paragraphs 0087 and 0006).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Kim et al. (US 2021/0124909) in view of Zheng et al. (US 2024/0185590).
Consider claim 16, and as applied to claim 12 above, Kim et al. further teaches that the portion of the memory is a second portion of the memory (i.e. “2 Level” of the volatile memory (130), see claim 1 rationale), and one or more processors (140) are to: retrieve a data structure (i.e. the object model) corresponding to the machine vision model from a first portion of the memory (i.e. from a non-volatile memory (120) of the memory, paragraph 0049); and deploy the machine vision model (600, 610) on the second portion of the memory (130) using the retrieved data structure (see paragraphs 0099 and 0049).
Kim et al. does not explicitly teach that the data structure includes a plurality of weights and biases.
Zheng et al. similarly teaches obtaining an object detection model which may be used to detect objects in an image (see paragraph 0054).
However, Zheng et al. additionally teaches that the object detection model includes a plurality of weights and biases (“The terminal may update parameters (e.g., weight and bias) of the object detection model to improve an accuracy of the object detection model.” paragraph 0125).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have the object model taught by Kim et al. include a plurality of weights and biases as taught by Zheng et al. for the benefit of improving accuracy of object detection (Zheng et al., paragraph 0125).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Youn et al. (US 2022/0101005) teaches a device (100, figure 1) including a processor (configuration controller, 120) that has an object detection model selector (125) and a sampling rate selector (127), see figure 3.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALBERT H CUTLER whose telephone number is (571)270-1460. The examiner can normally be reached approximately Mon - Fri 8:00-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached at (571)272-7564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALBERT H CUTLER/Primary Examiner, Art Unit 2637
Prosecution Insights