DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-2, 4, 7-15, 19 and 20 are pending. Claims 1 and 11 are amended. Claims 3, 5, 6, 16, 17 and 18 are cancelled.
Response to Arguments
Applicant's arguments filed on April 15, 2026 with respect to rejection of claims under 35 U.S.C. 103 has been fully considered; but they are not found persuasive. Specifically, in page 9 of its reply, Applicant argues in fifth paragraph that Zobel only discloses a parameter of ‘out of focus’ related to image quality, and therefore, Zobel does not teach multiple variable parameters. Examiner respectfully disagrees. Zobel teaches multiple parameters related to image quality in ¶0077: “any suitable processing technique to increase the image quality of an image. For example, the device 400 may ensure a consistent color balance, consistent luminance, continuous edges, consistent focus, consistent gamma, etc. in generating the final image (such as performing edge enhancement filters, blurring filters, noise reduction filters, etc., in generating a final image”. Accordingly, one of ordinary skill in the art would be motivated to combine the known elements and compare a tradeoff of quality metrics in order to determine an optimal solution. Therefore, applicant’s arguments are not found persuasive.
Consequently, THIS ACTION IS MADE FINAL.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 4, 5, 11, 12 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zobel (US 2021/0027439 A1) in view of Amine (Multiobjective Simulated Annealing: Principles and Algorithm Variants).
Regarding claim 1, Zobel teaches, A photographing system, (Zobel, ¶0001: “image capture systems and devices”) comprising: a plurality of cameras, photographing a scene for producing a plurality of sub-images; (Zobel, ¶0029: “two or more cameras from different perspectives may capture frames concurrently”) and a processor, signal-connected with the cameras for obtaining the sub-images, (Zobel, ¶0042: “the camera controller 410 (such as the image signal processor 412) may adjust or instruct the cameras 401 and 402”) wherein the processor is configured to analyze the sub-images for obtaining a plurality of objects contained in the scene, (Zobel, ¶0044: “capture frames of a scene from different perspectives. As a result, objects in the scene (such as people, background, etc.) are depicted with different orientations in associated frames from the cameras”) and the processor is further configured to establish a Pareto set of each of the objects, (Zobel, ¶0068: “determine the final orientation of each object of interest, and generate the final image”) the processor is further configured to splice the objects according to the Pareto set of each of the objects for generating an image (Zobel, ¶0077: “combining portions of different frames to generate a final image”) after a fusion of the sub-images, (Zobel, ¶0042: “combining frames from the first camera 401 and the second camera 402”) wherein each of the objects comprises a sub-object, (Zobel, ¶0037: “the first camera 401 and the second camera 402 have an overlapping field of view”; applicant’s specification ¶0027: “object 01 may be photographed by the cameras 100A, 100B, and 100C, so that the object 01 includes three sub-objects”, therefore, each sub-object is interpreted as object viewed from a different camera) and the processor is further configured to collect imaging feedback parameters (Zobel, ¶0066: “the device 400 may compare an image quality or other metrics for the segments in addition to the orientation of the object”) corresponding to all the sub-objects included in each of the objects in different sub-images from each of the objects of the scene (Zobel, ¶0066: “if the object appears out of focus or is obstructed in one depiction, the device 400 may prevent including the associated segment in the final image”) and calculate optimized optical parameters according to respective optical parameters of the cameras, (Zobel, ¶0042: “the camera controller 410 (such as the image signal processor 412) may adjust or instruct the cameras 401 and 402 to adjust one or more camera settings or configurations (such as the focal length, ISO setting, flash, resolution”) wherein the sub-object is a range of an image in which each of the objects appears in one of the sub-images, (Zobel, ¶0037: “the first camera 401 and the second camera 402 may be part of a multiple camera system for stitching, stacking, or comparing image frames of a scene (such as frozen moment visual effects or for increasing a field of view”) and the processor is further configured to establish the Pareto set of each of the objects by using (Zobel, ¶0066: “if the object appears out of focus or is obstructed in one depiction, the device 400 may prevent including the associated segment in the final image”) (Zobel, ¶0066: “the device 400 may compare an image quality or other metrics for the segments in addition to the orientation of the object. For example, if the object appears out of focus or is obstructed in one depiction, the device 400 may prevent including the associated segment”) and the optimized optical parameters corresponding to the sub-object, (Zobel, ¶0077: “device 400 may perform any suitable processing technique to increase the image quality of an image. For example, the device 400 may ensure a consistent color balance, consistent luminance, continuous edges, consistent focus, consistent gamma, etc. in generating the final image”) wherein the processor is further configured to fuse sub-objects of each of the objects that fall within the Pareto set (Zobel, ¶0029: “combine portions of the concurrently captured frames to include the desired orientation of each object (such as two people, a person and a background, etc.) in the scene in generating a final image”) and discard sub-objects that do not fall within the Pareto set for forming a plurality of fused images of the objects, (Zobel, ¶0066: “if the object appears out of focus or is obstructed in one depiction, the device 400 may prevent including the associated segment”) and splices the fused images of the objects for generating the image after a fusion of the sub-images, (Zobel, ¶0042: “combining frames from the first camera 401 and the second camera 402”) wherein each of the imaging feedback parameters (Zobel, ¶0066: “the device 400 may compare an image quality or other metrics for the segments”) comprises an image quality indicator (Zobel, ¶0077: “increase the image quality of an image. For example, the device 400 may ensure a consistent color balance, consistent luminance, continuous edges, consistent focus, consistent gamma”) and an imaging position indicator. (Zobel, ¶0050: “Based on the known positioning and distance of the cameras, the device 400 may associate regions of frames captured”). However, Zobel does not explicitly teach, a multi-objective simulated annealing algorithm.
In an analogous field of endeavor, Amine teaches, a multi-objective simulated annealing algorithm (Amine, page 01, col 1, ¶01: “multiobjective simulated annealing has been introduced with the aim to allow a construction of near-Pareto optimal solutions”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zobel using the teachings of Amine to introduce a multi-objective simulated annealing algorithm. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of comparing a tradeoff of quality metrics in order to determine an optimal set of objects to be included in the final image. Therefore, it would have been obvious to combine the analogous arts Zobel and Amine to obtain the invention in claim 1.
Regarding claim 4, Zobel in view of Amine teaches, The photographing system as claimed in claim 1, wherein the optical parameters of each of the cameras comprise an aperture, a focal length, a sensitivity, a white balance, or a resolution. (Zobel, ¶0042: “adjust one or more camera settings or configurations (such as the focal length, ISO setting, flash, resolution”).
Regarding claim 5, Zobel in view of Amine teaches, The photographing system as claimed in claim 1, wherein each of the imaging feedback parameters comprises an image quality indicator (Zobel, ¶0055: “an image quality metric between the frames 602 and 604”) and an imaging position indicator. (Zobel, ¶0050: “Based on the known positioning and distance of the cameras, the device 400 may associate regions of frames captured”).
Regarding claim 11, it recites a method with steps corresponding to the elements of the method recited in claim 1. Therefore, the recited steps of the method claim 11 are mapped to the proposed combination in the same manner as the corresponding elements of the system claim 1. Additionally, the rationale and motivation to combine Zobel and Amine presented in rejection of claim 1, apply to this claim.
Regarding claim 12, it recites a method with steps corresponding to the elements of the method recited in claim 4. Therefore, the recited steps of the method claim 12 are mapped to the proposed combination in the same manner as the corresponding elements of the system claim 4. Additionally, the rationale and motivation to combine Zobel and Amine presented in rejection of claim 1, apply to this claim.
Regarding claim 17, it recites a method with steps corresponding to the elements of the method recited in claim 5. Therefore, the recited steps of the method claim 17 are mapped to the proposed combination in the same manner as the corresponding elements of the system claim 5. Additionally, the rationale and motivation to combine Zobel and Amine presented in rejection of claim 1, apply to this claim.
Claims 2, 9, 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Zobel (US 2021/0027439 A1) in view of Amine (Multiobjective Simulated Annealing: Principles and Algorithm Variants) and in further view of Cuban et al. (US 2019/0122082 A1).
Regarding claim 2, Zobel in view of Amine teaches, The photographing system as claimed in claim 1, wherein the processor is further configured to analyze the sub-images by using a panoptic segmentation algorithm for (Zobel, ¶0065: “Any suitable means for segmentation may be used (such as using the example segmentation shapes and methods described herein), including automatic segmentation by the device 400”). However, the combination of Zobel and Amine does not explicitly teach, obtaining the objects included in each of the sub-images and their boundaries, and numbers the objects according to object types of the objects included in the scene for obtaining the objects included in the scene.
In an analogous field of endeavor, Cuban teaches, obtaining the objects included in each of the sub-images and their boundaries, (Cuban, ¶0041: “object recognition process can detect objects… determine a location of each person, such as by determining a boundary”) and numbers the objects according to object types of the objects (Cuban, ¶0043: “each person (or other type of object of interest) can also be assigned a unique identifier 608 that can be used to distinguish that object”) included in the scene for obtaining the objects included in the scene. (Cuban, ¶0056: “a person and an umbrella may be detected in the scene, which may indicate that it is a rainy day”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zobel in view of Amine using the teachings of Cuban to introduce detecting the boundary of each object. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of distinguishing each unique object. Therefore, it would have been obvious to combine the analogous arts Zobel, Amine and Cuban to obtain the invention of claim 2.
Regarding claim 9, Zobel and in view of Amine teaches, The photographing system as claimed in claim 1. However, the combination of Zobel and Amine does not explicitly teach wherein each of the cameras is a photosensor of complementary metal-oxide semiconductor, or a photosensor of charge coupled devices.
In an analogous field of endeavor, Cuban teaches, wherein each of the cameras is a photosensor of complementary metal-oxide semiconductor, or a photosensor of charge coupled devices. (Cuban, ¶0038: “The cameras can include any appropriate camera, as may include a complementary metal-oxide-semiconductor (CMOS)”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zobel in view of Amine using the teachings of Cuban to introduce a complementary metal-oxide-semiconductor (CMOS) cameras. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of higher efficiency in industrial and commercial applications. Therefore, it would have been obvious to combine the analogous arts Zobel, Amine and Cuban to obtain the invention in claim 9.
Regarding claim 13, it recites a method with steps corresponding to the elements of the method recited in claim 2. Therefore, the recited steps of the method claim 13 are mapped to the proposed combination in the same manner as the corresponding elements of the system claim 2. Additionally, the rationale and motivation to combine Zobel, Amine and Cuban presented in rejection of claim 2, apply to this claim.
Regarding claim 14, it recites a method with steps corresponding to the elements of the method recited in claim 9. Therefore, the recited steps of the method claim 14 are mapped to the proposed combination in the same manner as the corresponding elements of the system claim 9. Additionally, the rationale and motivation to combine Zobel, Amine and Cuban presented in rejection of claim 9, apply to this claim.
Claims 7, 8, 10, 15, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zobel (US 2021/0027439 A1), in further view of Amine (Multiobjective Simulated Annealing: Principles and Algorithm Variants) and still in further view of Zhang et al. (US 2009/0169102 A1).
Regarding claim 7, Zobel in view of Amine teaches, The photographing system as claimed in claim 1, wherein the processor is further configured to use (Zobel, ¶0074: “the device 400 may align a center position of the depiction 1002 with a center position of the associated depiction). However, the combination of Zobel and Amine does not explicitly teach a non-rigid alignment algorithm and then selects a fusion method according to an object type of each of the objects, and fuses the sub-objects of each of the objects that fall within the Pareto set according to the alignment base and the fusion method.
In an analogous field of endeavor, Zhang teaches, a non-rigid alignment algorithm (Zhang, ¶0015: “The preprocessed images may be warped to align the preprocessed images to each other”) and then selects a fusion method (Zhang, ¶0087: “fusion selection process to guarantee proper stitching”) according to an object type of each of the objects, and fuses the sub-objects of each of the objects that fall within the Pareto set (Zhang, ¶0117: “the winner among a plurality of images to be fused is not solely based on the amplitude or energy of a Laplacian pixel at (x,y) (better known as feature-selective fusion), but on a localized correlation measure (or salience measure) around (x,y)”) according to the alignment base and the fusion method. (Zhang, ¶0009: “performing pixel-level fusion on the Laplacian pyramid images based on a local salience measure that reduces aliasing artifacts to produce one salience-selected Laplacian pyramid image”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zobel in view of Amine using the teachings of Zhang to introduce selection of a fusion method. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of optimized image fusion. Therefore, it would have been obvious to combine the analogous arts Zobel, Amine and Zhang to obtain the invention in claim 7.
Regarding claim 8, Zobel in view of Amine and in further view of Zhang teaches, The photographing system as claimed in claim 7, wherein the fusion method comprises discrete wavelet transform, uniform rational filter bank, or Laplacian pyramid. (Zhang, ¶0006: “Laplacian pyramid fusion on the other hand provides excellent automatic selection of the important image detail for every pixel from multiple images at multiple image resolutions”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zobel in view of Amine and in further view of Zhang using the additional teachings of Zhang to introduce Laplacian pyramid fusion. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of optimized image fusion. Therefore, it would have been obvious to combine the analogous arts Zobel, Amine and Zhang to obtain the invention in claim 8.
Regarding claim 10, Zobel in view of Amine teaches, The photographing system as claimed in claim 1. However, the combination of Zobel and Amine does not explicitly teach wherein the optical parameters of each of the cameras comprise a full well capacity, a saturation capacity, a temporal dark noise, a dynamic range, a quantum efficiency, or a K-factor.
In an analogous field of endeavor, Zhang teaches, wherein the optical parameters of each of the cameras comprise a full well capacity, a saturation capacity, a temporal dark noise, a dynamic range, a quantum efficiency, or a K-factor. (Zhang, ¶0005: “combining video from two cameras that have a different aperture setting, providing significantly enhanced dynamic range”).
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Zobel in view of Amine using the teachings of Zhang to introduce cameras with a dynamic range. A person skilled in the art would be motivated to combine the known elements as described above and achieve the predictable result of capturing a wide range of darkest to brightest tones from a scene. Therefore, it would have been obvious to combine the analogous arts Zobel, Amine and Zhang to obtain the invention in claim 10.
Regarding claim 15, it recites a method with steps corresponding to the elements of the method recited in claim 10. Therefore, the recited steps of the method claim 15 are mapped to the proposed combination in the same manner as the corresponding elements of the system claim 10. Additionally, the rationale and motivation to combine Zobel, Amine and Zhang presented in rejection of claim 10, apply to this claim.
Regarding claim 19, it recites a method with steps corresponding to the elements of the method recited in claim 7. Therefore, the recited steps of the method claim 19 are mapped to the proposed combination in the same manner as the corresponding elements of the system claim 7. Additionally, the rationale and motivation to combine Zobel, Amine and Zhang presented in rejection of claim 7, apply to this claim.
Regarding claim 20, it recites a method with steps corresponding to the elements of the method recited in claim 8. Therefore, the recited steps of the method claim 20 are mapped to the proposed combination in the same manner as the corresponding elements of the system claim 8. Additionally, the rationale and motivation to combine Zobel, Amine and Zhang presented in rejection of claim 8, apply to this claim.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MEHRAZUL ISLAM whose telephone number is (571)270-0489. The examiner can normally be reached Monday-Friday: 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Saini Amandeep can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MEHRAZUL ISLAM/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights