Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-10, 14-19, 21-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Talbert et al. US2021/0274076 and further in view of Kojima US2022/0133140.
For claim 1, Talbert discloses “A system comprising:
an emitter (102; fig 1, 7E; 0062) comprising a plurality of electromagnetic sources that are emitted according to a variable pulse cycle (fig 7E shows variable pulses Pulse 1-4; embodiment described at 0171);
an image sensor (122; fig 1; 0064) comprising a pixel array that detects electromagnetic radiation;
a controller (124; fig 1; 0070) that synchronizes operations of the emitter and the image sensor;
wherein the controller instructs the image sensor to accumulate electromagnetic radiation and read out data according to a variable frame cycle comprising a plurality of frame periods (fig 4B, 7E; 0041, 0073); and
wherein a duration of the plurality of frame periods is adjustable on a per-frame basis (fig 4B, 7E; 0171)”.
wherein the controller provides data (“provides data” is considered to read on and include the recitation of dependent claim “the data that microcontroller writes”) regarding the variable frame cycle, wherein the data is determined based upon the variable pulse cycle of the emitter (fig 7E shows the variable frame cycle; 0172-0173 describes the variable emitter and corresponding sensor cycles, including the sensor being adjusted, i.e. provided a control signal via the controller);
Talbert does not disclose:
“a microcontroller”
wherein the controller provides data “to the microcontroller” regarding the variable frame cycle, wherein the data is determined based upon the variable pulse cycle of the emitter (see f. above);
“wherein the microcontroller provides instructions to the image sensor on a per-frame-basis to dynamically adjust a duration of the plurality of frame periods in the variable frame cycle”.
Kojima teaches in the same field of endeavor, a medical observation system which variably controls light emission duration and corresponding image sensor frames on a frame-by-frame basis (fig 6; 0083). Kojima also teaches this device as a modular system with attachable and detachable components (fig 1), including a camera head 5 with an imaging section 502 and camera head control section 504 as the claimed “microcontroller” (fig 2). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the modification of Kojima into the invention of Talbert in order to configure the system e.g. as claimed, in particular providing a microcontroller for the imaging section in the context of a modular system because it provides the added functionality of a modular system allowing separate components to be replaced independently if needed.
For claim 2, Talbert discloses “The system of claim 1, wherein each of the plurality of frame periods comprises: a blanking period wherein the pixel array accumulates the electromagnetic radiation; and a readout period wherein the pixel array reads out data for generating a data frame (fig 7E; 0073, 0090)”.
For claim 3, Talbert discloses “The system of claim 2, wherein the controller instructs the emitter to actuate the plurality of electromagnetic sources to pulse according to a variable illumination cycle comprising independent pulses of two or more different wavebands of electromagnetic radiation (fig 7E; 0061, 0171);
wherein each pulse within the variable illumination cycle of the emitter corresponds with a frame period of the plurality of frame periods of the image sensor (fig 7E; 0103); and
wherein the controller determines the pixel array’s efficiency in detecting the two or more different wavebands of electromagnetic radiation that are pulsed in the variable illumination cycle (fig 4B, 7E; 0169-0171)”.
For claim 4, Talbert discloses “The system of claim 3, wherein the controller calculates an optimized blanking period duration for each of the plurality of frame periods based on the corresponding pulse within the variable illumination cycle and the pixel array’s efficiency in detecting a waveband of the corresponding pulse (fig 4B, 7E; 0097, 0171)”.
For claim 5, Talbert discloses “The system of claim 1, wherein the controller is configured to synchronize the operations of the emitter and the image sensor by: instructing the emitter to pulse a first waveband of electromagnetic radiation during a first blanking period of the pixel array; and optimizing the duration of the first blanking period of the pixel array based on the pixel array’s efficiency in detecting the first waveband of electromagnetic radiation (fig 4B, 7E; 0097, 0171)”.
For claim 6, Talbert discloses “The system of claim 5, wherein the controller optimizes the duration of the first blanking period to enable the pixel array to accumulate a sufficient amount of energy to output data for generating a first data frame that corresponds with the first waveband of electromagnetic radiation such that the first data frame comprises a threshold degree of exposure for visualizing a scene (fig 4B, 7E; 0097, 0171)”.
For claim 7, Talbert discloses “The system of claim 1, wherein the controller instructs the emitter to pulse a first waveband of electromagnetic radiation during a first blanking period of the pixel array, and wherein: the controller lengthens a duration of the first blanking period if the pixel array’s sensitivity to the first waveband of electromagnetic radiation falls below a threshold efficiency; and the controller shortens a duration of the first blanking period if the pixel array’s sensitivity to the first waveband of electromagnetic radiation exceeds the threshold efficiency (fig 4B, 7E; 0097, 0171)”.
For claim 8, Talbert discloses “The system of claim 1, wherein the controller adjusts the duration of at least a portion of the plurality of frame periods to compensate for the pixel array comprising varying sensitivity to detecting different wavelengths of electromagnetic radiation emitted by the emitter (fig 4B, 7E; 0097, 0171)”.
For claim 9, Talbert discloses “The system of claim 1, wherein the plurality of electromagnetic sources comprises a visible source that pulses a visible wavelength of electromagnetic radiation, and wherein the visible source comprises one or more of: a white light source; or a red light source, a green light source, and a blue light source that are pulsed simultaneously or sequentially (fig 2; 0061, 0064, 0070-0071)”.
For claim 10, Talbert discloses “The system of claim 1,
wherein the plurality of electromagnetic sources comprises a multispectral source that pulses a spectral waveband of electromagnetic radiation, and wherein the multispectral source comprises (fig 1; 0064):
a first multispectral source that pulses electromagnetic radiation within a first narrowband of a visible waveband of the electromagnetic spectrum, wherein the first narrowband is 20 nm wide or less (0069 describes RGB pulses, i.e. red; 0071, 0135);
a second multispectral source that pulses electromagnetic radiation within a second narrowband of the visible waveband of the electromagnetic spectrum, wherein the second narrowband is 20 nm wide or less (0069 describes RGB pulses, i.e. green; 0071, 0135); and
a third multispectral source that pulses electromagnetic radiation within a near infrared waveband of the electromagnetic spectrum (0247 describes near IR emitter; 0071, 0135)”
a fluorescence source (0071 describes emitter 202 may be used to emit any wavelength of electromagnetic energy including fluorescence excitation wavelengths, e.g. 0120 lists some suitable wavelengths) that pulses a fluorescence excitation wavelength of electromagnetic radiation comprising one or more of:
a first fluorescence source that pulses electromagnetic radiation within a waveband from about 770 nm to about 795 nm (0071 describes emitter 202 may be used to emit any wavelength of electromagnetic energy including fluorescence excitation wavelengths, e.g. 0120 lists some suitable wavelengths, including 770 nm and 790 nm); or
a second fluorescence source that pulses electromagnetic radiation within a waveband from about 790 nm to about 815 nm (0071 describes emitter 202 may be used to emit any wavelength of electromagnetic energy including fluorescence excitation wavelengths, e.g. 0120 lists some suitable wavelengths, including 790 nm and 815 nm); and
a mapping source (0235, 0252 describes a laser scanning emitter for mapping) that pulses electromagnetic radiation in a mapping pattern, wherein the pixel array detects reflected electromagnetic radiation and outputs mapping data in response to the emitter pulsing the mapping pattern, and wherein the mapping data comprises information for determining one or more of a topographical map of a scene, a dimension of one or more objects within the scene, or a distance.
For claim 14, Talbert discloses “The system of claim 1, wherein the controller instructs the emitter to selectively cycle the plurality of electromagnetic sources according to a variable illumination cycle, and wherein the variable illumination cycle is adjustable based on user input (0109)”.
For claim 15, Talbert discloses “The system of claim 14, wherein the user input comprises an indication the image sensor should output color imaging and machine vision imaging, and wherein the controller adjusts the variable illumination cycle to comprise a pattern comprising: a plurality of pulses of a visible wavelength of electromagnetic radiation, wherein the image sensor outputs color imaging data in response to the emitter pulsing the visible wavelength of electromagnetic radiation; and a plurality of pulses of a machine vision emission, wherein the image sensor outputs machine vision imaging data in response to the emitter pulsing the machine vision emission (0109)”.
For claim 16, Talbert discloses “The system of claim 15, wherein the machine vision imaging comprises one or more of multispectral imaging, fluorescence imaging, or topographical mapping, and wherein the machine vision emission comprises a wavelength or pattern selected for the one or more of the multispectral imaging, the fluorescence imaging, or the topographical mapping (0109)”.
For claim 17, Talbert discloses “The system of claim 1, wherein the controller reprograms the image sensor prior to each frame period of the plurality of frame periods to set a blanking period duration for an upcoming frame period (0171, 0263)”.
For claim 18, Talbert discloses “The system of claim 1, further comprising a memory buffer (1504; fig 15; 0212), and wherein:
the image sensor outputs a plurality of datasets corresponding with the plurality of frame periods (fig 1; 0067-0070), and wherein each of the plurality of datasets comprises information for generating a data frame (0067);
two or more of the plurality of frame periods of the frame cycle comprise different blanking period durations (316; fig 7E; 0103, 0171); and
the image sensor outputs the plurality of datasets to the memory buffer at irregular intervals due to the frame cycle comprising the different blanking period durations (126; fig 1; and blanking period 316; fig 7E; 0103, 0171, 0067-0070)”.
For claim 19, Talbert discloses “The system of claim 18, further comprising an image processing pipeline that receives the plurality of datasets stored in the memory buffer and processes the plurality of datasets at regular intervals (316; fig 7E; 0067-0070, 0103, 0171).
For claim 21, modified Talbert discloses “The system of claim 1, wherein the data that the image sensor receives from the microcontroller (Kojima: camera head control section 504; fig 2) is used to set the duration for an upcoming frame period of the plurality of frame periods (Talbert: fig 7E shows the variable frame cycle; 0172-0173 describes the variable emitter and corresponding sensor cycles, including the sensor being adjusted, i.e. provided a control signal via the controller)”.
For claim 22, modified Talbert discloses “The system of claim 1, wherein the data the microcontroller (Kojima: camera head control section 504; fig 2) writes (writing the data is considered to read on the parent claim recitation of “provides data” and vice versa, otherwise the writing of the data would lack antecedent basis) to the image sensor is based at least in part on a type of an upcoming data frame (Talbert: fig 4B, 7E shows the variable frame cycle; 0041, 0073, 0172-0173 describes the variable emitter and corresponding sensor cycles, including the sensor being adjusted, i.e. provided a control signal via the controller, i.e. “the sensor and/or emitter are adjusted to compensate for differences in energy values for the pulsed partitioned spectrums of light”)”.
For claim 23, modified Talbert discloses “The system of claim 1, wherein the data the microcontroller writes (writing the data is considered to read on the parent claim recitation of “provides data” and vice versa, otherwise the writing of the data would lack antecedent basis) to the image sensor is based at least in part on feedback analyzed from a previously read out data frame (Talbert: 0173 describes adjusting the sensor and recording a frame based on readings from the adjusted sensor, i.e. a histogram of a previous frame is obtained at 902 in fig 9, and the sensor is adjusted based on sensitivity)”.
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Talbert and Kojima as applied to claim 1 above, and further in view of Choi et al. US2021/0068975.
For claim 20, Talbert does not disclose “The system of claim 1, wherein the microcontroller writes data to the image sensor to set a blanking period duration for each of the plurality of frame periods, and wherein two or more of the plurality of frame periods comprise a different blanking period duration”, but does disclose various computer storage media types used in the device [0075], including RAM, ROM, and “other types of memory”, not specifying sensor registers; while [0172] describes the control/adjustment of the sensor which requires transmission of instructions and data for changing the frame periods. Additionally, Kojima discloses the microcontroller for the image sensor (fig 6; 0083). Choi teaches in the same field of endeavor, computer memory for storing instructions and data can comprise RAM, various caches, and registers [0071]. Since Talbert fails to disclose the nature of the computer memory as a register but allows for “other types of memory”, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have used any suitable computer memory known in the art, including the one taught by Choi, to achieve the predictable result of storing and retrieving computer data.
Response to Arguments
Applicant’s arguments with respect to claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See PTO892.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAE K WOO whose telephone number is (571)272-0837. The examiner can normally be reached M-F 8:30-2:30p, 6p-9p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Anhtuan Nguyen can be reached at (571) 272-4963. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jae Woo/Examiner, Art Unit 3795
/ANH TUAN T NGUYEN/Supervisory Patent Examiner, Art Unit 3795 08/15/25
Prosecution Insights