Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 10,794,710) in view of Choi et al. (US 2023/0011755 A1) in view of Yang et al. (CN 114445261 A).
Regarding claim 1, Liu discloses an imaging system comprising:
a plurality of cameras, wherein each camera of the plurality of cameras comprises at least one image sensor (See Fig 2 Cameras 1-4 and Col 4 line 5-35);
a plurality of video links (See Fig 2 and Fig 3), wherein each of the plurality of video links is configured to:
connect a particular camera of the plurality of cameras to a central processor (See Col 9 line 20 – Col 10 line 45 control unit receiving image data from cameras), and
a memory, wherein the memory stores instructions for processing image data obtained from the plurality of cameras (See Col 7 and 9-32 control unit including memory); and
the central processor, wherein the central processor is configured to execute the instructions to perform a method for processing the image data (See Col 9 line 20 – Col 10 line 45), the method comprising:
aggregating the image data for each camera of the plurality of cameras to produce aggregated image data (combining the time synchronized images to create a single view, see Col 9 line 45-67); and
processing the aggregated image data to produce a set of at least one environmental image (See Col 9 line 45-Col 10 line 67 the aggregated images are rectified to a common image plane resulting in a 360-degrees view. The resulting image is corrected to achieve reliable computer vision.).
Liu does not explicitly disclose carrying power, from the central processor to the particular camera.
Choi carry power, from the central processor to the particular camera (See [0155-0156]);
Prior to the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to modify the known system of Liu with the known methods of Choi predictably resulting in carrying power, from the central processor to the particular camera by applying the court recognized rational of applying a known technique to a known device (method, or product) ready for improvement to yield predictable results. The modification would have the benefit of controlling the power at the various cameras as suggested by Choi.
Liu does not explicitly disclose aggregating the image data for each camera of the plurality of cameras to produce aggregated image data, using at least one mobile industry processor interface (MIPI) aggregator
Yang discloses that it was known to aggregate image data using an MIPI (“As shown in FIG. 1, it also can be connected in series coaxial cable between the serialized chip 121 and the integrated signal processor 122 to expand the data transmission length of the camera. The multi-path camera camera the MIPI-CSI or DVP signal after serializing through coaxial cable transmission, the transmission distance of the camera can be expanded to more than 10 m, it is convenient to realize the access of the multi-path camera.
According to another specific embodiment of the present application, the serialized chip 121 and the data acquisition interface 110 adopt GPIO synchronous pulse for data transmission; and/or, the serialized chip 121 and the integrated signal processor 122 adopt the Frame Sync synchronous pulse for data transmission.
As shown in FIG. 3, the serialized chip 121 synchronous acquisition is realized by GPIO pulse, DeserializerHub supports external GPIO synchronization pulse and internal frame synchronous pulse Frame Sync mode, the same pulse signal is synchronously transmitted to the 4-path Serializer chip chip camera Adopting the data transmission device 100, it can realize the expansion of 4-way camera on the basis of maintaining the original hardware platform, supporting MIPI-CSI and DVP two camera, at the same time, the aggregator support multi-camera hardware frame synchronous control, camera the long distance transmission, The invention can quickly realize multiple-camera input expansion of the robot.
In summary, the embodiment of the data transmission device, the 2 to 4 paths camera signal to aggregate MIPI-CSI interface, the MIPI-CSI camera interface of 1 path extends out 4 paths of camera interface, the original hardware platform based on the four-path camera expansion, at the same time, supporting multi-camera hardware frame synchronous control, camera the long distance transmission, can quickly realize robot multi camera input expansion, avoiding the robot hardware platform redesign, shortening the hardware development period, realizing fast expansion upgrade of the robot product, The invention greatly simplifies the transmission mode and cost of the multi-camera to the image receiving device.”).
Prior to the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to modify the combination further with the known methods of Yang predictably resulting in carrying power, from the central processor to the particular camera by applying the court recognized rational of applying a known technique to a known device (method, or product) ready for improvement to yield predictable results. The modification would have the benefit of controlling the power at the various cameras as suggested by Choi.
Regarding claim 2, Liu Choi and Yang further disclose the imaging system of claim 1, wherein the method further comprises applying the set of at least one environmental image to machine vision-based navigation (See Liu Col 5 line 9-55 autonomous vehicle using a 3D map for navigation).
Regarding claim 3, Liu Choi and Yang further disclose the imaging system of claim 1, wherein a particular camera of the plurality of cameras is configured to produce part of the image data by aggregating captured sensor data from each image sensor of the at least one image sensor included in the particular camera (See Choi [0100] where a master camera may generate an output image based on image data received from slaves devices).
Regarding claim 4, Liu Choi and Yang further disclose the imaging system of claim 3, wherein aggregating the captured sensor data is performed by a particular MIPI aggregator included in the particular camera (See Yang “ a serialized chip is connected with a camera through a data collecting interface, all serialization chip aggregate connected to an integrated signal processor. serial aggregator the image data received by the data collecting interface and converting into serial signal, and then sending the serial signal to the integrated signal processor.”).
Claim(s) 5-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 10,794,710) in view of Choi et al. (US 2023/0011755 A1) in view of Yang et al. (CN 114445261 A) in further view of Trank et al. (US 20220147042 A1).
Regarding claim 5, Liu Choi and Yang disclose the imaging system of claim 3, but do not disclose wherein a given video link of the plurality of video links comprises a Gigabit Multimedia Serial Link (GMSL).
Trank discloses that it was known to collect video stream from multiple cameras (See [0012]) and to collect the image data via MIPI extended GMSL serializer/deserializer)
Prior to the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to modify the combination with the known methods of Trank predictably resulting in a given video link of the plurality of video links comprises a Gigabit Multimedia Serial Link (GMSL) by applying the court recognized rational of applying a known technique to a known device (method, or product) ready for improvement to yield predictable results. The modification would have the benefit of having improved computer vision capabilities, reduce number of wires, and power consumption as suggested by Trank.
Regarding claim 6, Liu Choi Yang and Trank further disclose the imaging system of claim 5, wherein aggregating the captured sensor data is performed by a serializer of the GMSL (See Trank [0053] [0058]).
Regarding claim 7, Liu Choi Yang and Trank further disclose the imaging system of claim 6, wherein: the central processor comprises a camera expansion board; and the camera expansion board comprises at least one deserializer coupled with the serializer (See Trank [0021] [0023] [0053] serializer/deserializer board installed [0058]).
Regarding claim 8, Liu Choi Yang and Trank further disclose the imaging system of claim 7, wherein an even number of video links, of the plurality of video links, are attached to a given deserializer of the at least one deserializer (See Trank [0053] [0058]).
Regarding claim 9, Liu Choi Yang and Trank further disclose the imaging system of claim 8, wherein the even number of video links are attached to the given deserializer using a singular serial interface (See Yang Fig 2 multi-camera serialization into an integrated signal processor).
Regarding claim 10, Liu Choi Yang and Trank further disclose the imaging system of claim 9, wherein the singular serial interface is a MIPI bus (See Trank [0021] [0023] [0053] [0058]).
Regarding claim 11, Liu Choi Yang and Trank further disclose the imaging system of claim 8, wherein the even number of video links are used to aggregate image data, from corresponding cameras of the plurality of cameras, into a shared video stream (See Yang Fig 2 even number of cameras aggregated to serializer “The data transmission device claimed by the invention, system and method, the data transmission device is used for accessing multi-channel camera wherein the data transmission device comprises: at least two data collecting interfaces and serial aggregator, each data collecting interface is used for externally connecting a data end of a camera; the serial aggregator comprises a plurality of serialized chip and an integrated signal processor, the input end of each serialized chip is connected with a data collecting interface,”).
Regarding claim 12, Liu Choi Yang and Trank further disclose the imaging system of claim 11, wherein the shared video stream is configured using stereo vision (See Trank [0048] collecting stereovision data, stereo cameras).
Regarding claim 13, Liu Choi Yang and Trank further disclose the imaging system of claim 7, wherein the at least one deserializer is used to integrate at least one additional non-camera sensor into the imaging system (See Liu Fig 2 IMU, GPS, See Trank [0046] [0048] GPS).
Regarding claim 14, Liu Choi Yang and Trank further disclose the imaging system of claim 13, wherein the at least one deserializer is used to apply the at least one additional non-camera sensor to a navigation objective (See Liu [0003] detection of objects and navigation capabilities).
Regarding claim 15, Liu Choi Yang and Trank further disclose the imaging system of claim 13, wherein the at least one additional non-camera sensor comprises at least one of:an inertial measurement unit (IMU); or a three-axis gyroscope (See Liu Fig 2 IMU).
Regarding claim 16, Liu Choi Yang and Trank further disclose the imaging system of claim 13, wherein the method further comprises synchronizing a capture of the at least one additional non-camera sensor with captures of at least some of the plurality of cameras (See Liu Col 9 line 45- Col 10 line 10 images are synchronized to internal clock as well as auxiliary sensors see Col 8 line 20-51, Col 9 line 1-15).
Regarding claim 17, Liu Choi Yang and Trank further disclose the imaging system of claim 16, wherein synchronizing the capture of the at least one additional non-camera sensor with the captures of the at least some of the plurality of cameras comprises transmitting a pulse width modulation pulse (See Yang “As shown in FIG. 3, the serialized chip 121 synchronous acquisition is realized by GPIO pulse, DeserializerHub supports external GPIO synchronization pulse and internal frame synchronous pulse Frame Sync mode, the same pulse signal is synchronously transmitted to the 4-path Serializer chip chip camera”).
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 10,794,710) in view of Choi et al. (US 2023/0011755 A1) in view of Yang et al. (CN 114445261 A) in further view of Haneda (US 20070097224 A1).
Regarding claim 18, Liu Choi and Yang disclose the imaging system of claim 1, wherein: each image sensor of the at least one image sensor, included in a given camera of the plurality of cameras, comprises a personal frequency reference clock; and the personal frequency reference clock is monitored by a corresponding video link for the given camera.
Haneda discloses that it was known for a camera (See [0026] device includes camera) to perform synchronization (See [0015] [0049] [0053] [0066]).
Prior to the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to modify the combination with the known methods of Haneda predictably resulting in each image sensor of the at least one image sensor, included in a given camera of the plurality of cameras, comprises a personal frequency reference clock; and the personal frequency reference clock is monitored by a corresponding video link for the given camera. by applying the court recognized rational of applying a known technique to a known device (method, or product) ready for improvement to yield predictable results. The modification would have the benefit of synchronizing multiple cameras as suggested by Haneda.
Claim(s) 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et al. (US 10,794,710) in view of Choi et al. (US 2023/0011755 A1) in view of Yang et al. (CN 114445261 A) in further view of Olsson et al. (US 2017/0023492 A1).
Regarding claim 19, Liu Choi and Yang disclose the imaging system of claim 1, wherein a given image sensor of the at least one image sensor, included in a given camera of the plurality of cameras, comprises a personal image signal processor (ISP).
Olsson discloses an image sensor includes a image processor (See [0237]).
Prior to the effective filing date of the invention it would have been obvious to one of ordinary skill in the art to modify the combination with the known methods of Olsson predictably resulting in a given image sensor of the at least one image sensor, included in a given camera of the plurality of cameras, comprises a personal image signal processor (ISP) by applying the court recognized rational of applying a known technique to a known device (method, or product) ready for improvement to yield predictable results. The modification would have the benefit of electrically connecting elements.
Regarding claim 20, Liu Choi Yang and Olsson further disclose the imaging system of claim 19, wherein the personal ISP and the given image sensor are appended to a shared rigid flex printed circuit board (PCB) appended to the given camera (See Olsson [0237] one or more PCBs).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to FERNANDO ALCON whose telephone number is (571)270-5668. The examiner can normally be reached Monday-Friday, 9:00am-7:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Pendleton can be reached at (571)272-7527. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
FERNANDO . ALCON
Examiner
Art Unit 2425
/FERNANDO ALCON/Primary Examiner, Art Unit 2425
Prosecution Insights