DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Office Action is in response to the application filed on 12/16/2024. Claims 1-15 are pending and have been examined.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 12/16/2024 filed in accordance with the provisions of 37 CFR 1.97. Accordingly, it is being considered by the examiner.
Allowable Subject Matter
Claims 4-7, and 12-15 are objected to as depending from a rejected base claim. However, these claims would be allowable if rewritten in independent form to include all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C.
102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction
of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status
The following is a quotation of 35 U.S.C. 103 which forms the basis for all
obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention
is not identically disclosed as set forth in section 102, if the differences between the claimed
invention and the prior art are such that the claimed invention as a whole would have been obvious
before the effective filing date of the claimed invention to a person having ordinary skill in the art
to which the claimed invention pertains. Patentability shall not be negated by the manner in which
the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35
U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness
Claims 1-2, 8, and 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over DING, Xu (CN-110827314-A) (translation provided and citation given from the translated document) hereinafter “Ding” in view of Jiang et al (US-20230281919-A1) hereinafter “Jiang”.
Regarding Claim 1 Ding-Jiang
Ding discloses 1. (Original) A multispectral multi-sensor synergistic processing method, Single (Ding, [0001]”a single target tracking methods and related equipment” [0067] “the first and second images are composed of red, green, and blue channels”) comprising:
acquiring images of a plurality of channels, wherein an image of each channel is a monochrome image, and colors of images of different channels are different; (Ding teaches determining “a first color channel of the first image and the second color channel of the second image”(¶[0065]). Ding further teaches “the first and second images are composed of red, green, and blue channels. (¶[0067]), and further teaches synthesizing a target image using those channels “The target image is synthesized by superimposing the first color channel and the second color channel.” (¶[0159])).
performing detection, identification and tracking of a target according to monochrome images, generating a tracking box for the target and storing information of the tracking box; (Ding, [0050] “Step 201: The electronic device acquires a first image and a second image in the same target video file, and acquires a first tracking box in the first image, wherein the first image is a preset frame image preceding the second image, the first image and the second image include the same tracking target, and the first tracking box is used to select the tracking target in the first image.” [0054] “The target video file is a video file captured while tracking the target. The target video file is stored on an electronic device or in the cloud”.) and
However, in the same field of endeavor TTT discloses more explicitly the following:
registering and fusing the images of the plurality of channels to generate a fused image, and overlaying stored information of the tracking box onto the fused image. (Jiang, [0059] “the coordinate transformed luminance image and depth image are channel stacked with the grayscale image, to form a fused image having information of three channels, as shown in FIG. 3d. .. to align information of the two-dimensional image with that of the three-dimensional image in coordinates, and store color information, luminance information, and depth information, etc. for each coordinate in the image,…the color information, the luminance information, and the depth information may alternatively be fused by weighting to obtain an image having information of a single channel or multiple channels.” )
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the application to modify the teachings of Ding with Jiang to incorporate Jiang’s techniques for registering images, fusing channels to generate a fused image, and overlaying information on the fused image because “Such fused image data may provide more information for subsequent deep learning model analysis, thereby obtaining a more accurate detection result.”(Jiang,[0012])
Regarding Claim 2 Ding-Jiang
Ding-Jiang discloses 2. (Original) The method according to claim 1,
wherein the overlaying stored information of the tracking box onto the fused image. (Ding, [0050] “…acquires a first tracking box in the first image, wherein the first image is a preset frame image preceding the second image, the first image and the second image include the same tracking target, and the first tracking box is used to select the tracking target in the first image.” [0054] “The target video file is a video file captured while tracking the target. The target video file is stored on an electronic device or in the cloud”.) comprises:
overlaying stored latest information of the tracking box onto the fused image. (Ding [0131] “The one or more programs are stored in the memory and configured to be executed by the processor. The programs include instructions for performing the following steps:” [0132] “A first image and a second image are obtained from the same target video file, and a first tracking box is obtained from the first image, wherein the first image is a preset frame image preceding the second image, the first image and the second image include the same tracking target, and the first tracking box is used to select the tracking target in the first image [0133] “The first image and the second image are superimposed to obtain the target image;”)
Regarding Claim 8 Ding-Jiang
Ding-Jiang discloses 8. (Currently Amended) A multispectral multi-sensor synergistic processing apparatus (Ding, [0013] “….provide a single-target tracking apparatus, the apparatus comprising…”) , comprising a memory and a processor, and the memory storing a program, wherein the program, when being read and executed by the processor, implements the multispectral multi-sensor synergistic processing method (Ding, [0019] “provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing the steps of the method described in the first aspect of …”) according to any one of
Regarding Claim 10 Ding-Jiang
Ding-Jiang discloses 10. (Currently Amended) A non-transitory computer-readable storage medium storing one or more programs, wherein the one or more programs is executable by one or more processors to implement the multispectral multi-sensor synergistic processing method (Ding, [0021] “this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, the computer program being operable to cause a computer to perform some or all of the steps described in the method described in the first aspect of embodiments of this application. This computer program product can be a software installation package.” according to any one of claim 1.
Regarding Claim 11 Ding-Jiang
Ding-Jiang discloses 11. (New) The method according to claim 5, further comprising: in response to determining to select monochrome images of a plurality of channels to perform detection, identification and tracking of the target, taking a combined set of targets detected in the monochrome images of the plurality of channels as targets of the fused image, wherein the overlaying stored information of the tracking box onto the fused image comprises:
overlaying stored information of tracking boxes of targets detected in the monochrome images of the plurality of channels onto the fused image. (Ding, [0050] “Step 201: The electronic device acquires a first image and a second image in the same target video file, and acquires a first tracking box in the first image, wherein the first image is a preset frame image preceding the second image, the first image and the second image include the same tracking target, and the first tracking box is used to select the tracking target in the first image.” [0054] “The target video file is a video file captured while tracking the target. The target video file is stored on an electronic device or in the cloud”.)
Claim Rejections - 35 USC § 103
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Ding-Jiang in view of Tang (US 20120200734-A1) hereinafter “Tang”.
Regarding Claim 3 Ding-Jiang-Tang
Ding-Jiang discloses 3. (Original) The method according to claim 1, wherein the channel comprises a red channel (Ding, [0067] “the first and second images are composed of red, green, and blue channels”);
the performing detection, identification and tracking of a target according to monochrome images (Ding, [0050] “Step 201: The electronic device acquires a first image and a second image in the same target video file, and acquires a first tracking box in the first image, wherein the first image is a preset frame image preceding the second image, the first image and the second image include the same tracking target, and the first tracking box is used to select the tracking target in the first image.” [0054] “The target video file is a video file captured while tracking the target. The target video file is stored on an electronic device or in the cloud”.) comprises:
Ding-Jiang does not expressly disclose
in response to determining that current ambient light intensity is less than a preset light intensity threshold, selecting a near-infrared image of the red channel for detection, identification and tracking of the target.
However, in the same field of endeavor Tang discloses more explicitly the following:
in response to determining that current ambient light intensity is less than a preset light intensity threshold, selecting a near-infrared image of the red channel for detection, identification and tracking of the target. (Tang,[0021]” for controlling a camera unit to generate a processed digital image represented by a plurality of image pixels. The method includes detecting an intensity of ambient light in a vicinity of the camera unit, …includes raw color image data and raw infrared image data…adapting the camera unit to a mode of operation selected from a plurality of modes of operation defined for the camera unit in relation to the detected ambient light intensity. The plurality of modes of operation comprising a first mode of operation and a second mode of operation.” [0129] “ a mode of operation for the camera unit is selected based upon the detected conditions of the ambient light, such as the intensity or color temperature of the ambient light.” [0131] “…the low light mode may be selected when the detected ambient light intensity is between the lower and upper ambient light intensity thresholds.”)
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the application to modify the teachings of Ding-Jiang with Tang to create the system of Ding-Jiang as outlined above in order to select a near-infrared image of the red channel for detection, identification and tracking of the target in response to determining that current ambient light intensity is less than threshold as suggested by Tang. One of ordinary skill in the art would have been motivated to incorporate Tang’s into Ding-Jiang in order to improve target detection and tracking in low-light environments by utilizing near-infrared information available through the red channel , thereby enhancing the spectral selectivity of the color pixels. ( Tang, [0086])
Claim Rejections - 35 USC § 103
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Ding-Jiang in view of McLean et al (US-20090021598-A1) hereinafter “McLean”.
Regarding Claim 9 Ding-Jiang-McLean
Ding-Jiang discloses The apparatus according to claim 8,
Ding-Jiang does not explicitly disclose
further comprising a lens, a beam splitter prism and a plurality of sensors, wherein, the lens is configured to receive external light and transmit the external light to the beam splitter prism;
the beam splitter prism is configured to split incident light to obtain a plurality of monochromatic lights, and the plurality of monochromatic lights are incident on the plurality of sensors respectively, wherein each of the plurality of monochromatic lights is incident on one sensor; and
the sensor is configured to convert the incident light into an electrical signal and output the electrical signal to the processor.
However, in the same field of endeavor McLean discloses more explicitly the following:
further comprising a lens (McLean, [0251] “Fig. 11 “Imaging lens,), a beam splitter prism (McLean, [0257] “an image-splitter prism 21, 44a-d (FIGS. 2 and 3) “ and a plurality of sensors [0087] “when the sensor array has plural sensor…”) wherein, the lens is configured to receive external light and transmit the external light to the beam splitter prism; (McLean teaches incoming radiation entering the optical system and being directed to the compound/splitter prism. (¶¶[0038]-[0039))
the beam splitter prism (McLean, [0090] Fig. 3 “splitter prism” ) is configured to split incident light to obtain a plurality of monochromatic lights, and the plurality of monochromatic lights are incident on the plurality of sensors respectively, wherein each of the plurality of monochromatic lights is incident on one sensor; (McLean, [0039] “Following the optics 45, in multispectral applications the radiation 48' entering the compound prism 44a-b-c-d is split to form four output beams 49a-b-c-d, conventionally passed through color filters 42, as noted above, to form red 49a, green 49b, blue 49c and infrared 49d beams. In known multipolarization systems the beams are passed through polarization filters instead, to form beams of differently oriented polarization.” See also ¶[0258]-[0259] ) and
the sensor is configured to convert the incident light into an electrical signal and output the electrical signal to the processor. (McLean teaches multispectral imaging arrays that receive the incident beams and generate image data (Electrical image signals) which are provided to a data-acquisition-and-control subsystem and a real-time processing. ¶[0254]-0255] and ¶[0261]-[0262]))
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the application to modify the teachings of Ding-Jiang with McLean to create the system of Ding-Jiang as outlined above in order to further comprise a beam splitter prism and a plurality of sensor, where the beam splitter prism splits light and the sensor convert the incident light into electrical signals, as suggested by McLean. One of ordinary skill in the art would have been motivated to incorporate McLean’s into Ding-Jiang in order to enhance radiometric efficiency of the multispectral imaging system. (McLean, [0049])
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
XIE, Jian-bin. CN-109740563-A
Investigation of Registration and Deblurring Method for Multispectral Image. (2018)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ASTEWAYE GETTU ZEWEDE whose telephone number is (703)756-1441. The examiner can normally be reached Mo-Fr 8:30 am to 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached at (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ASTEWAYE GETTU ZEWEDE/Examiner, Art Unit 2481
/WILLIAM C VAUGHN JR/Supervisory Patent Examiner, Art Unit 2481
Prosecution Insights