DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5, 7, and 16-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 5 recites the limitation "the AR overlay information" in line 12. There is insufficient antecedent basis for this limitation in the claim. The only claim that recites “(AR) overlay information” is claim 3 which claim 5 is not dependent from. Thus it is unclear as to whether the claim is mean to be dependent upon a different claim or if AR overlay information was supposed to be previously recited in some manner. In the interest of compact prosecution the Examiner will interpret the claim as if it recites “to add augmented reality (AR) overlay information” which would render the claim definite.
Regarding claim 7, claim 7 recites dependence on cancelled claim 6, such that the claim lacks antecedent basis for its terminology. Furthermore, line 2 of the claim recites “the phosphor screen” without any antecedent basis. Furthermore as the claim elements could be connected to various previous claims it is not clear which claim would necessarily be the claim which is the parent. In the interest of compact prosecution, Claim 5 has been chosen to provide antecedent basis as the “phosphor screen” is recognized as related to an element corresponding to image intensifier tubes (IIT’s). Furthermore, “the phosphor screen” will be interpreted as “a phosphor screen.”
Claim 16 recites the limitation "the pass-through electronics" in line 6. There is insufficient antecedent basis for this limitation in the claim. The claim does not previously recite any pass-through electronics making it unclear as to whether it was meant to be introduced in some form but was not or whether some earlier limitations are meant to be interpreted as pass-through electronics or whether pass-through electronics are supposed to be considered an inherent part of such a digital imaging device. In the interest of compact prosecution the Examiner will interpret the claim as if it recites “with pass-through electronics” such that pass-through electronics are introduced and then further defined as in the claims which would render the claim definite.
Dependent claims 17-21 are also rejected for carrying through this deficiency of the parent claims.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, 5, 7 and 16-18, is/are rejected under 35 U.S.C. 102(a)(1)/(a)(2) as being anticipated by Dobbie et al1 (“Dobbie”).
Regarding claim 1, Dobbie teaches a digital image device comprising (note this device is addressed by addressing the limitations below which together constitute such a device):
a digital image sensor configured to receive an image and to generate digital image data corresponding to the image (see Dobbie, column 8, lines 55-67 through column 9, lines 1-28 and figure 13 teaching “an image intensified camera module” and “objective lens 90 focuses light from the scene onto the photocathode of the image intensifier 92” and “image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96” and “CMOS camera functional block has the purpose of sensing the 2-D image on its pixel array and generating a real-time video signal representation of that image” where “the CMOS camera may output digital video, analog video, or both signals” such that here the image intensified camera module is a digital image sensor configured as claimed which receives photons of light that comprise an image and senses the image for “generating a real-time video signal representation of that image” where that may be “digital video”);
a digital display configured to receive the digital image data and to generate a displayed image corresponding to the digital image data (see Dobbie, column 7, lines 1-62 and figures 9 and 10 teaching a “video display (HMD)” where the digital video signal as generated above by the image intensified camera module “is transmitted to the video display module 72, which receives the video signal, and generates a two dimensional image that can be viewed by the human eye” such that here the digital display is configured to receive that digital image data and generate a displayed image corresponding to the digital image data );
pass-through electronics configured to receive the digital image data directly from an output interface of the digital image sensor, the output interface for providing digital image data, and to provide the digital image data to the digital display without performing data conversion processing on the digital image data (see Dobbie, column 10, lines 64-67 through column 12, lines 1-8 and figure 16, teaching “image processing subsystem within a video enhanced night vision device may physically be comprised of dedicated electronic circuits” where “Such a subsystem is shown as the “image processor” block in FIGS. 16” and “the processor receives the raw video signal from the camera subsystem, and sends the processed video signal to the display” where “As the data stream is sent to the image processor subsystem, it is written directly into one of the two video frame buffers 122 and 124” where “writing process is managed by the control logic portion 126 of the field programmable gate array (FPGA)128” and “control logic manages the overall timing and operation of the image processing subsystem by monitoring the timing information embedded in the input video stream and by monitoring the system clock 130” such that here “frame buffers” function as pass-through electronics that receive the digital image data directly from the output interface of the digital image sensor as further explained in column 8, lines 55-67 through column 9, lines 1-28 and figure 13 teaching “an image intensified camera module” and “objective lens 90 focuses light from the scene onto the photocathode of the image intensifier 92” and “image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96” and “CMOS camera functional block has the purpose of sensing the 2-D image on its pixel array and generating a real-time video signal representation of that image” where “the CMOS camera may output digital video, analog video, or both signals” such that here the digital image sensor outputs the digital image data as “digital video” which is digital image data and where the output interface of the digital image sensor corresponds to the output CMOS camera interface which outputs the “digital video” image data and provides it directly to the pass-through electronics as explained above as in column 10, lines 64-67 through column 12, lines 1-8 and figure 16 where “FIG. 16 shows the functional architecture of an embodiment of the image processing module subsystem. This figure illustrates the blocks that reside within the “image processing” block of FIG. 10. The input to this block is the digital video signal from the camera subsystem” such that the output interface from the camera subsystem inputs that digital video signal to the image processing block and does so through pass through electronics in the form of the frame buffers explained above which receive the digital image data directly from the output interface and as the buffers merely store the data and rely on other processors external to them to modify any data therein then the pass-through electronics are configured to provide the digital image data to the digital display without performing data conversion on the digital image data); and
an external electronics processor in communication with the pass-through electronics (note that the claim recites an “external electronics processor” but fails to define what exactly external is with reference to, where something that is external to something else is anything that is located, seen, or used on the outside or surface of something or is otherwise outside some other element and thus for example if an “electronics processor” configured as claimed is external to any other element such that it is not part of that specific element then it is considered external to the element and for example external does not require any aspect of the electronics processor to be physically separated, or remote or wireless; thus see Dobbie, column 10, lines 64-67 through column 12, lines 1-8 and figure 16, teaching “image processing subsystem within a video enhanced night vision device may physically be comprised of dedicated electronic circuits” where “Such a subsystem is shown as the “image processor” block in FIGS. 16” and “the processor receives the raw video signal from the camera subsystem, and sends the processed video signal to the display” where “As the data stream is sent to the image processor subsystem, it is written directly into one of the two video frame buffers 122 and 124” where “writing process is managed by the control logic portion 126 of the field programmable gate array (FPGA)128” and “control logic manages the overall timing and operation of the image processing subsystem by monitoring the timing information embedded in the input video stream and by monitoring the system clock 130” where “arithmetic logic unit (ALU) 132 within the FPGA performs the actual image processing operations. The frame buffers are loaded with image data in the form of numerical values that correspond to pixel brightness values. The fundamental principle of image processing is to modify these values and produce an enhanced image for display. For each image processing function (contrast stretching, edge detection, etc.) there is a specific algorithm, or program, that must be performed. The control logic loads this program from a program RAM (not seen in FIG. 16) into the FPGA and uses the ALU to execute the algorithm. Essentially, the ALU numerically manipulates the pixel data in one of the frame buffers in accordance with the program logic. The specific image processing algorithms for different functions are well known to those skilled in the art” such that here “frame buffers” function as pass-through electronics which are in communication with an electronics processor such as the ALU 132 of the FPGA 128 which are both instances of external electronics processors in communication with these pass-through electronics which simply pass through the data that they receive or are given according to the timing of the input and output where “As a video frame from the camera is being written into one of the frame buffers, the other frame buffer is outputting a processed video frame to the display. To maintain a real-time video image for the user, the output video stream must consist of continuously updated, processed images. The control logic 126 manages this timing. Each frame buffer is thus alternately a receiver of data from the camera and a source of data for the display” and “ALU typically performs this math on a video frame immediately after it is written into the frame buffer and prior to the start of frame transmission to the display. By maintaining this timing sequence, the image processor subsystem is able to supply real-time video data to the display”);
wherein the pass-through electronics are configured to generate a copy of the digital image data and to provide the copy of the digital image data to the external electronics processor (Dobbie, column 10, lines 64-67 through column 12, lines 1-8 and figure 16 where “FIG. 16 shows the functional architecture of an embodiment of the image processing module subsystem. This figure illustrates the blocks that reside within the “image processing” block of FIG. 10. The input to this block is the digital video signal from the camera subsystem” and “As the data stream is sent to the image processor subsystem, it is written directly into one of the two video frame buffers 122 and 124” such that this generates a copy of the digital image data and “frame buffers are loaded with image data in the form of numerical values that correspond to pixel brightness values. The fundamental principle of image processing is to modify these values and produce an enhanced image for display. For each image processing function (contrast stretching, edge detection, etc.) there is a specific algorithm, or program, that must be performed. The control logic loads this program from a program RAM (not seen in FIG. 16) into the FPGA and uses the ALU to execute the algorithm. Essentially, the ALU numerically manipulates the pixel data in one of the frame buffers in accordance with the program logic. The specific image processing algorithms for different functions are well known to those skilled in the art” such that here the frame buffers provide the external electronics processor the copy of the digital image data), and
the external electronics processor is configured to generate additional data based on the copy of the digital image data and communicate the additional data to the pass-through electronics for display on the digital display (see Dobbie, column 10, lines 64-67 through column 12, lines 1-8 and figure 16 where “FIG. 16 shows the functional architecture of an embodiment of the image processing module subsystem. This figure illustrates the blocks that reside within the “image processing” block of FIG. 10. The input to this block is the digital video signal from the camera subsystem” and “frame buffers are loaded with image data in the form of numerical values that correspond to pixel brightness values. The fundamental principle of image processing is to modify these values and produce an enhanced image for display. For each image processing function (contrast stretching, edge detection, etc.) there is a specific algorithm, or program, that must be performed. The control logic loads this program from a program RAM (not seen in FIG. 16) into the FPGA and uses the ALU to execute the algorithm. Essentially, the ALU numerically manipulates the pixel data in one of the frame buffers in accordance with the program logic. The specific image processing algorithms for different functions are well known to those skilled in the art” where these modifications are generations of additional data based on the copy of the data from the frame buffers and then this additional data in the form of the modification is communicated back to the pass-through electronics framebuffers for display on the digital display as where “a video frame from the camera is being written into one of the frame buffers, the other frame buffer is outputting a processed video frame to the display. To maintain a real-time video image for the user, the output video stream must consist of continuously updated, processed images. The control logic 126 manages this timing. Each frame buffer is thus alternately a receiver of data from the camera and a source of data for the display. This scheme is sometimes referred to a “ping pong” buffer because of the continuously alternating roles played by each buffer” and “ALU typically performs this math on a video frame immediately after it is written into the frame buffer and prior to the start of frame transmission to the display. By maintaining this timing sequence, the image processor subsystem is able to supply real-time video data to the display”).
Regarding claim 2, Dobbie teaches all that is required as applied to claim 1 above and further teaches a first control clock (see Dobbie, column 11, lines 18-67 through column 12, lines 1-8 and figure 16 teaching “control logic manages the overall timing and operation of the image processing subsystem by monitoring the timing information embedded in the input video stream and by monitoring the system clock 130. The clock provides clock signals (CLK) to the FPGA control logic, the frame buffers, and to the camera subsystem. This master timing signal is needed to properly synchronize the overall video signal train and ensure that image processor doesn't lose image frames or otherwise corrupt the system timing”), wherein: a functional resolution of the digital image sensor is equal to a functional resolution of the digital display (note that a “functional resolution” is considered any resolution that a sensor or image or display functions to provide, and importantly note that as recited the claim is not limited to a situation in which the native resolutions of a sensor or display are equal, though that would be encompassed within the broader reasonable interpretation that for example the functional resolution of a sensor is equal to a functional resolution of a digital display means that the resolution which the sensor functions with may be made equal to the resolution of the digital display where such making of the functional resolution to be equal would then mean the functional resolutions of the digital image sensor and digital display are equal; see Dobbie, column 8, lines 55-67 through column 10, lines 1-23 and figures 13 and 14, teaching “fiber optic taper, or minifier 100, is optically bonded to the flat 102 of the screen fiberoptic 104 of image intensifier tube 106, The small end of the taper, with its minified image, is bonded to the IC imaging array chip 108 which is itself bonded to the chip carrier 110. In this manner, the intensified image present on the tube screen is coherently transferred and scaled to the array. If the array is used with a diagonal dimension close to the tube imaging format diameter, the fiber optic taper can be eliminated, and the solid state array can be bonded directly to the tube screen optic” such that here the resolution of the sensor corresponds to the IC imaging array chip 108 where this array corresponds to the pixel array values to be displayed which are passed to the pass through electronics frame buffers as in column 11, lines 18-67 through column 12, lines 1-8 teaching “the data stream is sent to the image processor subsystem, it is written directly into one of the two video frame buffers 122 and 124” and “frame buffers are loaded with image data in the form of numerical values that correspond to pixel brightness values” and “For each image processing function (contrast stretching, edge detection, etc.) there is a specific algorithm, or program, that must be performed. The control logic loads this program from a program RAM (not seen in FIG. 16) into the FPGA and uses the ALU to execute the algorithm. Essentially, the ALU numerically manipulates the pixel data in one of the frame buffers in accordance with the program logic” such that in accordance with a mode, the data may be manipulated but need not be necessarily be converted as in column 7, lines 37-62 and figure 10 for example teaching “camera subsystem generates the “raw” (unprocessed) video signal. This signal is connected to either the image processor module or directly to the video display module. The user makes this selection via the selection switch 82” such that here the user may select to not provide any image processing functions and the raw unprocessed video signal may be passed directly to the video display module such that this means that as no processing need be performed that the functional resolution of the digital image sensor output is equal to the functional resolution of the digital display); and
the digital image sensor and the digital display both operate using the first control clock (see Dobbie, column 11, lines 18-67 through column 12, lines 1-8 and figure 16 teaching “control logic manages the overall timing and operation of the image processing subsystem by monitoring the timing information embedded in the input video stream and by monitoring the system clock 130. The clock provides clock signals (CLK) to the FPGA control logic, the frame buffers, and to the camera subsystem. This master timing signal is needed to properly synchronize the overall video signal train and ensure that image processor doesn't lose image frames or otherwise corrupt the system timing”).
Regarding claim 3, Dobbie teaches all that is required as applied to claim 1 above and further teaches wherein the pass-through electronics are configured to add information to the digital image data, wherein the information comprises one or more of color information, brightness information, calibration information, and augmented reality (AR) overlay information (note that the claim does not limit or define how the pass through electronics are configured to add information to the digital image data and for example such addition could occur in connection with another element that provides information that the pass-through electronics are configured to accommodate or pass through which would add such information to the digital image data; see Dobbie, column 11, lines 18-56 teaching “frame buffers are loaded with image data in the form of numerical values that correspond to pixel brightness values. The fundamental principle of image processing is to modify these values and produce an enhanced image for display. For each image processing function (contrast stretching, edge detection, etc.) there is a specific algorithm, or program, that must be performed. The control logic loads this program from a program RAM (not seen in FIG. 16) into the FPGA and uses the ALU to execute the algorithm. Essentially, the ALU numerically manipulates the pixel data in one of the frame buffers in accordance with the program logic” as further explained in column 7, lines 47-62 teaching “image processor module 80 provides the capability of performing real-time image processing of the video signal for the purpose of enhancing the picture presented to the user. Such enhancements may include but are not limited to contrast stretching, edge detection/enhancement, MTF peaking (aperture correction), integration, mixing/overlaying of intensified video with externally input video, averaging, and other known image processing functions” where the processor is provided the pixel data for processing from the pass-through electronics frame buffers that are configured to add information to the digital image data by providing a copy of the sensor data to the external image processor to process the data in order to add the process pixel data back to the pass-through electronics frame buffer and the enhancement information that is added comprises color information through modifying pixel values which are color, brightness information through contrast stretching, and calibration information such as through MTF peaking or aperture correction which calibrates the image for the aperture).
Regarding claim 5, as rendered definite as explained above, Dobbie teaches all that is required as applied to claim 1 above and further teaches an image intensifier tube (lIT) configured to receive a scene image corresponding to an observed scene and to produce an enhanced image based upon the scene image and the digital image sensor is configured to receive the enhanced image (see Dobbie, column 8, lines 55-67 through column 9, lines 1-51 and figure 13 teaching “the objective lens 90 focuses light from the scene onto the photocathode of the image intensifier 92. The tube also contains a microchannel plate (MCP) for amplifying electrons and a phosphor screen having a screen optic 95. The image intensifier is preferably a late model version such as a Generation III, Generation IV, or later model when such becomes available. The tube is powered by an auto-gating HVPS 97. The auto-gate 94 controls the HVPS 97, which supplies voltage to the microchannel plate and screen, and also controls the gate driver 99 which supplies the cathode voltage” and “provide a properly exposed and adequately bright image from the intensifier tube, the control circuits automatically optimize the screen luminance from the tube. This image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96. As an example, FIG. 13 shows a CMOS “camera-on-a-chip” at this position in the architecture, although other solid state imaging arrays could also be used”);
wherein the external electronics processor is configured to determine: (note that the manner of determining or how such determining takes place is not limited by the claim language)
one or more dark portions of the digital image data corresponding to one or more portions of the enhanced image having an absence of light; one or more bright portions of digital image data corresponding to one or more portions of the enhanced image data having a brightness value corresponding to light; and the pass-through electronics are configured to add augmented reality (AR) overlay information to the one or more dark portions of the digital image data and to add one or more of color and brightness information to the one or more bright portions of the digital image data (see Dobbie, column 8, lines 1-42 teaching “Thermal imaging cameras are responsive to different portions of the infrared spectrum than image intensification devices, and thus provide additional information to the viewer. An advantage of the present invention over prior art direct view optical video night vision goggles is the ability to provide thermal imaging” and “the image from the thermal camera may be “fused” with the image from the image intensified camera, so that the viewer sees the two images superimposed on each other” and “the unprocessed camera and thermal camera video signals are fed to the image processing block 87 having fusion capability. The image processing block 87 may include the same image enhancement functions previously described, and also includes the image fusion function. The image fusion function electronically overlays the two video images, adjusts their relative brightnesses, and may also add color cueing information. The fusion and image enhancement functions are both controlled by the user via physical controls on the goggle. The video display presents the video image that is the result of the enhancements and fusion processing” such that here dark portions of an image having an absence of light corresponds to a scene being imaged in such conditions of low light which means that more light is absent than other conditions and if a bright portion is any portion having a brightness value then this is simply a registered image value for the pixel and AR information in the form of the thermal imagery and color information may be added to the digital image data by the controller and this would be via the pass-through electronics frame buffers that are configured to add the information as in columns 11-12 as explained previously).
Regarding claim 7, as rendered definite as explained above, Dobbie teaches all that is required as applied to claim 5 and further teaches wherein the digital image sensor comprises a CMOS sensor and a phosphor screen is deposited onto the CMOS sensor (note that as explained above a phosphor screen is known to be associated with an IIT as in claim 5; see Dobbie, column 8, lines 55-67 through column 9, lines 1-51 and figure 13 teaching “the objective lens 90 focuses light from the scene onto the photocathode of the image intensifier 92. The tube also contains a microchannel plate (MCP) for amplifying electrons and a phosphor screen having a screen optic 95” and “image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96. As an example, FIG. 13 shows a CMOS “camera-on-a-chip” at this position in the architecture”).
Regarding claim 16, Dobbie teaches a digital imaging device (note this device is addressed by addressing the limitations below which together constitute such a device) comprising:
a digital image sensor configured to receive photons comprising an image and to generate digital image data corresponding to the photons (see Dobbie, column 8, lines 55-67 through column 9, lines 1-28 and figure 13 teaching “an image intensified camera module” and “objective lens 90 focuses light from the scene onto the photocathode of the image intensifier 92” and “image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96” and “CMOS camera functional block has the purpose of sensing the 2-D image on its pixel array and generating a real-time video signal representation of that image” where “the CMOS camera may output digital video, analog video, or both signals” such that here the image intensified camera module is a digital image sensor configured as claimed which receives photons of light that comprise an image and senses the image for “generating a real-time video signal representation of that image” where that may be “digital video”);
a digital display configured to receive the digital image data and to generate a displayed image corresponding to the digital image data (see Dobbie, column 7, lines 1-62 and figures 9 and 10 teaching a “video display (HMD)” where the digital video signal as generated above by the image intensified camera module “is transmitted to the video display module 72, which receives the video signal, and generates a two dimensional image that can be viewed by the human eye” such that here the digital display is configured to receive that digital image data and generate a displayed image corresponding to the digital image data ); and
an external electronics processor in communication with pass-through electronics (note that the claim recites an “external electronics processor” but fails to define what exactly external is with reference to, where something that is external to something else is anything that is located, seen, or used on the outside or surface of something or is otherwise outside some other element and thus for example if an “electronics processor” configured as claimed is external to any other element such that it is not part of that specific element then it is considered external to the element and for example external does not require any aspect of the electronics processor to be physically separated, or remote or wireless; thus see Dobbie, column 10, lines 64-67 through column 12, lines 1-8 and figure 16, teaching “image processing subsystem within a video enhanced night vision device may physically be comprised of dedicated electronic circuits” where “Such a subsystem is shown as the “image processor” block in FIGS. 16” and “the processor receives the raw video signal from the camera subsystem, and sends the processed video signal to the display” where “As the data stream is sent to the image processor subsystem, it is written directly into one of the two video frame buffers 122 and 124” where “writing process is managed by the control logic portion 126 of the field programmable gate array (FPGA)128” and “control logic manages the overall timing and operation of the image processing subsystem by monitoring the timing information embedded in the input video stream and by monitoring the system clock 130” where “arithmetic logic unit (ALU) 132 within the FPGA performs the actual image processing operations. The frame buffers are loaded with image data in the form of numerical values that correspond to pixel brightness values. The fundamental principle of image processing is to modify these values and produce an enhanced image for display. For each image processing function (contrast stretching, edge detection, etc.) there is a specific algorithm, or program, that must be performed. The control logic loads this program from a program RAM (not seen in FIG. 16) into the FPGA and uses the ALU to execute the algorithm. Essentially, the ALU numerically manipulates the pixel data in one of the frame buffers in accordance with the program logic. The specific image processing algorithms for different functions are well known to those skilled in the art” such that here “frame buffers” function as pass-through electronics which are in communication with an electronics processor such as the ALU 132 of the FPGA 128 which are both instances of external electronics processors in communication with these pass-through electronics which simply pass through the data that they receive or are given according to the timing of the input and output where “As a video frame from the camera is being written into one of the frame buffers, the other frame buffer is outputting a processed video frame to the display. To maintain a real-time video image for the user, the output video stream must consist of continuously updated, processed images. The control logic 126 manages this timing. Each frame buffer is thus alternately a receiver of data from the camera and a source of data for the display” and “ALU typically performs this math on a video frame immediately after it is written into the frame buffer and prior to the start of frame transmission to the display. By maintaining this timing sequence, the image processor subsystem is able to supply real-time video data to the display” ),
wherein the digital image sensor is configured to output the digital image data from an output interface of the digital image sensor, the output interface for providing digital image data, and to provide the digital image data directly to the pass-through electronics (see Dobbie, column 8, lines 55-67 through column 9, lines 1-28 and figure 13 teaching “an image intensified camera module” and “objective lens 90 focuses light from the scene onto the photocathode of the image intensifier 92” and “image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96” and “CMOS camera functional block has the purpose of sensing the 2-D image on its pixel array and generating a real-time video signal representation of that image” where “the CMOS camera may output digital video, analog video, or both signals” such that here the digital image sensor outputs the digital image data as “digital video” which is digital image data and where the output interface of the digital image sensor corresponds to the output CMOS camera interface which outputs the “digital video” image data and provides it directly to the pass-through electronics as explained above as in column 10, lines 64-67 through column 12, lines 1-8 and figure 16 where “FIG. 16 shows the functional architecture of an embodiment of the image processing module subsystem. This figure illustrates the blocks that reside within the “image processing” block of FIG. 10. The input to this block is the digital video signal from the camera subsystem” such that the output interface from the camera subsystem inputs that digital video signal to the image processing block and does so through pass through electronics in the form of the frame buffers explained above which receive the digital image data directly from the output interface where “As the data stream is sent to the image processor subsystem, it is written directly into one of the two video frame buffers 122 and 124” and then the pass-through electronics pass this data through to an external electronics processor as explained below),
the pass-through electronics are configured to generate a copy of the digital image data and to provide the copy of the digital image data to the external electronics processor (see Dobbie, column 10, lines 64-67 through column 12, lines 1-8 and figure 16 where “FIG. 16 shows the functional architecture of an embodiment of the image processing module subsystem. This figure illustrates the blocks that reside within the “image processing” block of FIG. 10. The input to this block is the digital video signal from the camera subsystem” and “As the data stream is sent to the image processor subsystem, it is written directly into one of the two video frame buffers 122 and 124” such that this generates a copy of the digital image data and “frame buffers are loaded with image data in the form of numerical values that correspond to pixel brightness values. The fundamental principle of image processing is to modify these values and produce an enhanced image for display. For each image processing function (contrast stretching, edge detection, etc.) there is a specific algorithm, or program, that must be performed. The control logic loads this program from a program RAM (not seen in FIG. 16) into the FPGA and uses the ALU to execute the algorithm. Essentially, the ALU numerically manipulates the pixel data in one of the frame buffers in accordance with the program logic. The specific image processing algorithms for different functions are well known to those skilled in the art” such that here the frame buffers provide the external electronics processor the copy of the digital image data), and
the external electronics processor is configured to generate additional data based on the copy of the digital image data and to communicate the additional data to the pass-through electronics for display on the digital display (see Dobbie, column 10, lines 64-67 through column 12, lines 1-8 and figure 16 where “FIG. 16 shows the functional architecture of an embodiment of the image processing module subsystem. This figure illustrates the blocks that reside within the “image processing” block of FIG. 10. The input to this block is the digital video signal from the camera subsystem” and “frame buffers are loaded with image data in the form of numerical values that correspond to pixel brightness values. The fundamental principle of image processing is to modify these values and produce an enhanced image for display. For each image processing function (contrast stretching, edge detection, etc.) there is a specific algorithm, or program, that must be performed. The control logic loads this program from a program RAM (not seen in FIG. 16) into the FPGA and uses the ALU to execute the algorithm. Essentially, the ALU numerically manipulates the pixel data in one of the frame buffers in accordance with the program logic. The specific image processing algorithms for different functions are well known to those skilled in the art” where these modifications are generations of additional data based on the copy of the data from the frame buffers and then this additional data in the form of the modification is communicated back to the pass-through electronics framebuffers for display on the digital display as where “a video frame from the camera is being written into one of the frame buffers, the other frame buffer is outputting a processed video frame to the display. To maintain a real-time video image for the user, the output video stream must consist of continuously updated, processed images. The control logic 126 manages this timing. Each frame buffer is thus alternately a receiver of data from the camera and a source of data for the display. This scheme is sometimes referred to a “ping pong” buffer because of the continuously alternating roles played by each buffer” and “ALU typically performs this math on a video frame immediately after it is written into the frame buffer and prior to the start of frame transmission to the display. By maintaining this timing sequence, the image processor subsystem is able to supply real-time video data to the display”).
Regarding claim 17, Dobbie teaches all that is required as applied to claim 16 above and further teaches an image intensifier tube (IIT) configured to receive photons corresponding to a scene image, to produce an enhanced image corresponding to the scene image, and to provide the enhanced image to the digital image sensor (see Dobbie, column 8, lines 55-67 through column 9, lines 1-28 and figure 13 teaching “camera design may incorporate features, including the use of a 16-mm intensifier tube, a gated high voltage power supply (HVPS) which may be smaller than that used in stationary applications, and small, low power electronic imaging circuits” and “an image intensified camera module” and “objective lens 90 focuses light from the scene onto the photocathode of the image intensifier 92” seen as a “tube” and “image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96” and “CMOS camera functional block has the purpose of sensing the 2-D image on its pixel array and generating a real-time video signal representation of that image” where “the CMOS camera may output digital video, analog video, or both signals” such that here the image intensified camera module is a digital image sensor configured as claimed which receives photons of light from a scene image that comprise a scene image enhanced by the IIT which are provided to the digital image sensor such as the “CMOS” camera which takes that enhanced image and converts it to a “digital video” image stream).
Regarding claim 18, Dobbie teaches all that is required as applied to claim 16 above and further teaches optics comprising an optic lens for providing photons to the digital image sensor during daylight or bright light operation of the digital imaging device (see Dobbie, column 8, lines 55-67 through column 9, lines 1-28 and figure 13 teaching “objective lens 90 focuses light from the scene onto the photocathode of the image intensifier 92” and “tube is powered by an auto-gating HVPS 97. The auto-gate 94 controls the HVPS 97, which supplies voltage to the microchannel plate and screen, and also controls the gate driver 99 which supplies the cathode voltage. The auto-gating allows for operations into higher light levels than is possible with a conventional, non-gated wrap-around type HVPS. In the present architecture, the control loops that determine microchannel plate voltage and gate duty cycle may be integral to the HVPS. Auto-gate block 94 includes an automatic brightness control function (ABC), which truncates the linear gain characteristic of the tube and effectively sets a maximum brightness output. To provide a properly exposed and adequately bright image from the intensifier tube, the control circuits automatically optimize the screen luminance from the tube. This image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96” such that the optic lens would provide photons to the digital image sensor during daylight or bright light operations of the digital image device and for example could use the brightness control functions in daylight or bright light operations, where it should also be noted that these are extremely broad limitations almost to the point of subjectivity, but it is considered that at least the limitations define that in some manner the optics must be capable of operating as optics during times which are not explicitly night such that the sun is not contributing any light whatsoever).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dobbie in view of Vakil2.
Regarding claim 8, Dobbie teaches all that is required as applied to claim 1 above and further teaches wherein the digital image sensor is disposed on a first side of a circuit card, the digital display is disposed on a second side of the circuit card, the circuit card comprises the pass-through electronics, and the digital image data is passed through the circuit card from the digital image sensor to the digital display (see Dobbie, column 8, lines 55-67 through column 10, lines 1-57 teaching “image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96” where “the CMOS camera may output digital video” such that a digital sensor is disposed on a circuit card, and a digital display is disposed on a side of a circuit card which comprises the pass-through electronics as in column 6, lines 4-27 teaching “video signal is received by the display printed circuit board 60” and as in column 10, lines 64-67 through column 11, lines 1-65 “the data stream is sent to the image processor subsystem, it is written directly into one of the two video frame buffers 122 and 124. This writing process is managed by the control logic portion 126 of the field programmable gate array (FPGA)128” where this circuit comprises the pass-through electronics explained above which passes the digital image data from the digital image sensor to the digital display). Dobbie fails to teach the digital image sensor is disposed on a first side of a circuit card and the digital display is disposed on a second side of a circuit card, rather Dobbie directs the video display to other optics such that it is not necessary to have a circuit card with the sensor and display so disposed and as Dobbie only suggests use of a HMD form factor with an eyepiece. Thus Dobbie stands as a base device upon which the claimed invention can be seen as an improvement through allowing an arrangement that would allow for a compact form factor that combines the display and sensor disposed on sides of a circuit card.
In the same field of endeavor relating to arranging digital displays and digital image sensors with respect to a circuit card, Vakil teaches that it is known to dispose a digital image sensor on a first side of a circuit card, and to dispose a digital display on a second side of the circuit card and to pass through image data from the sensor to the digital display (see Vakil, paragraphs 0031-0033 and figure 1 teaching “first camera module 100 includes a camera 102 and a substrate 104” paragraphs 0054-0060 and figure 5A teaching a circuit card such as “substrate 104” aligned with “display 502” where it can be seen in figure 5b that the image sensor is disposed on a first side of the circuit card 104 and the digital display is disposed on a second side of the circuit card 104 and as in paragraphs 0094-0095 “a display unit (e.g., display unit 502) or some other component of the multi-media device is positioned in a space at least partly defined by the second side of the first substrate” which “advantageously creates a compact form factor and reduces the thickness of the multi-media device housing the third substrate (e.g., substrate 504), the coupled first camera module and second camera module, and the display unit (e.g., display unit 502)”). Thus Vakil teaches a known technique for image sensor and display integration applicable to the base system of Dobbie ready for improvements relating to integrating displays and image sensors.
Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to modify Dobbie by applying the known techniques of Vakil as doing so would be no more than application of a known technique to a base system ready for improvement which would yield predictable results and result in an improved system. The predictable result of the combination would be that Dobbie’s display and image sensor would be ensured as disposed along first and second sides of a circuit card such that the digital image sensor and display of Dobbie would utilize sides of a circuit card to achieve their similar function of taking in image data and converting it to display data for the digital display in order to display in a form factor such as in Vakil. This would result in an improved system as such integration “advantageously creates a compact form factor” as suggested by Vakil. Furthermore one of ordinary skill in the art would have been motivated to modify Dobbie using the teachings of Vakil as Vakil provides a method for integrating similar types of components and suggests the teachings “advantageously creates a compact form factor” (see Vakil, paragraph 0022).
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dobbie in view of Smith3.
Regarding claim 9, Dobbie teaches all that is required as applied to claim 1 above and further teaches the digital image sensor comprises an array of digital pixels, wherein a digital pixel of the array of digital pixels is configured to perform image sensing and analog-to-digital conversion of analog image data to generate the digital image data (see Dobbie, column 8, lines 55-67 through column 9, lines 1-51 and figure 13 teaching “image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96. As an example, FIG. 13 shows a CMOS “camera-on-a-chip” at this position in the architecture, although other solid state imaging arrays could also be used” and “CMOS camera functional block has the purpose of sensing the 2-D image on its pixel array and generating a real-time video signal representation of that image” and “CMOS camera may output digital video” such that here the analog image data from the image intensifier is converted to a digital pixel in the array of digital pixels to generate the digital image data );
the digital image sensor comprises a sensor array comprising a plurality of sensor pixels, the sensor array comprising a plurality of rows and a plurality of columns of sensor pixels (see Dobbie, column 8, lines 55-67 through column 9, lines 1-51 and figure 13 teaching “image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96. As an example, FIG. 13 shows a CMOS “camera-on-a-chip” at this position in the architecture, although other solid state imaging arrays could also be used” and “CMOS camera functional block has the purpose of sensing the 2-D image on its pixel array and generating a real-time video signal representation of that image” and “fiber optic taper, or minifier 100, is optically bonded to the flat 102 of the screen fiberoptic 104 of image intensifier tube 106, The small end of the taper, with its minified image, is bonded to the IC imaging array chip 108 which is itself bonded to the chip carrier 110. In this manner, the intensified image present on the tube screen is coherently transferred and scaled to the array. If the array is used with a diagonal dimension close to the tube imaging format diameter, the fiber optic taper can be eliminated, and the solid state array can be bonded directly to the tube screen optic” such that here the array is the CMOS sensor pixel array and comprises rows and columns of sensor pixels);
the digital display comprises a display array comprising a plurality of display pixels, the display array comprising a plurality of rows and a plurality of columns of display pixels (see Dobbie, column 6, lines 4-27 teaching the digital display where “video signal is fed to the display module” and “outputs electrical signals to the flat panel display 62, which converts the signals into a two dimensional video image” and as in column 11, lines 1-67 through column 12, lines 1-8 this display is fed the pixels after processing or can be fed the pixels raw directly where “As the data stream is sent to the image processor subsystem, it is written directly into one of the two video frame buffers 122 and 124” and “For each image processing function (contrast stretching, edge detection, etc.) there is a specific algorithm, or program, that must be performed. The control logic loads this program from a program RAM (not seen in FIG. 16) into the FPGA and uses the ALU to execute the algorithm. Essentially, the ALU numerically manipulates the pixel data in one of the frame buffers in accordance with the program logic” and “Each frame buffer is thus alternately a receiver of data from the camera and a source of data for the display. This scheme is sometimes referred to a “ping pong” buffer because of the continuously alternating roles played by each buffer” such that here the pixels from the camera correspond to the display pixels and have rows and columns);
wherein a number of rows of the plurality of rows of sensor pixels is the same as a number of rows of the plurality of rows of the display pixels, and a number of columns of the plurality of columns of sensor pixels is the same as a number of columns of the plurality of columns of the display pixels (see Dobbie, column 8, lines 55-67 through column 9, lines 1-51 and figure 13 teaching “image from the image intensifier is fiber optically coupled by screen fiber optic 95 to the imaging chip 96. As an example, FIG. 13 shows a CMOS “camera-on-a-chip” at this position in the architecture, although other solid state imaging arrays could also be used” and “CMOS camera functional block has the purpose of sensing the 2-D image on its pixel array and generating a real-time video signal representation of that image” and “fiber optic taper, or minifier 100, is optically bonded to the flat 102 of the screen fiberoptic 104 of image intensifier tube 106, The small end of the taper, with its minified image, is bonded to the IC imaging array chip 108 which is itself bonded to the chip carrier 110. In this manner, the intensified image present on the tube screen is coherently transferred and scaled to the array. If the array is used with a diagonal dimension close to the tube imaging format diameter, the fiber optic taper can be eliminated, and the solid state array can be bonded directly to the tube screen optic” and as in as in column 11, lines 1-67 through column 12, lines 1-8 this array is passed to the image process where “As the data stream is sent to the image processor subsystem, it is written directly into one of the two video frame buffers 122 and 124” and “Each frame buffer is thus alternately a receiver of data from the camera and a source of data for the display. This scheme is sometimes referred to a “ping pong” buffer because of the continuously alternating roles played by each buffer” such that here the number of rows and columns of the sensor and display are the same as they share the same rows and columns of the frame buffer);
the digital image sensor is configured to read out a first row of the plurality of rows of sensor pixels and to bin image data corresponding to the first row in to a first data packet; and the digital display is configured to receive the first data packet and to fill a first row of display pixels with data comprising the first data packet, the first row of display pixels corresponding to the first row of sensor pixels (see Dobbie, column, 11, lines 1-67 through column 12, lines 1-8 teaching as above “Each frame buffer is thus alternately a receiver of data from the camera and a source of data for the display. This scheme is sometimes referred to a “ping pong” buffer because of the continuously alternating roles played by each buffer” and “ALU typically performs this math on a video frame immediately after it is written into the frame buffer and prior to the start of frame transmission to the display. By maintaining this timing sequence, the image processor subsystem is able to supply real-time video data to the display” such that here there is reading out of sensor data and receiving of it by the display through the frame buffer).
Dobbie teaches all of the above but fails to detail the digital image sensor and digital display pixel formats with respect to rows and reading of rows according to some sort of basic row based binning and packet sending of row data from an image sensor to the digital display. Thus Dobbie stands as a base device upon which the claimed invention can be seen as an improvement by providing a readout and filling operation that could lead to more efficient operations such as through pipelining and more efficient data access and processing.
In the same field of endeavor relating to capturing image data and providing it for digital display, Smith teaches that it is known to provide a digital image sensor wherein the digital image sensor comprises an array of digital pixels, wherein a digital pixel of the array of digital pixels is configured to perform image sensing and analog-to-digital conversion of analog image data to generate the digital image data (see Smith, paragraphs 0039-0043 teaching “Night vision system 52…combines an addressable display within analog image intensifier 58” and “electrical signals from digital imager 54 are sent across an electrical bus 55 onto at least one and preferably two electron multipliers 61 a and 61 b” which are digitally addressable displays where “electrons emitted from each emitter can be electrically addressed by control circuitry corresponding to that emitter” and another identical “image intensifier 58b” is provided and “At the backside of the image intensifier 58 b is a digital sensor 56 b mounted on or separate from the backside surface of image intensifier tube 58 b. The digital sensor 56 b comprises a plurality of active or passive pixel sensor devices arranged in an array operating as optical pixels with CMOS circuity to convert the photons emitted from image intensifier 58 b to electrical signals, similar to the pixel array 40 shown in FIG. 3” where “Digital sensor 56 b can be a CMOS imager used as an active pixel sensor device or passive pixel device. Digital sensor 56 b can be a CMOS imager chip or die with integrated amplifiers as an active pixel sensor device that incorporates both the photodiode and a read out amplifier” and “the improved night vision system 52 of FIG. 6 has a digital channel dimension or width DCW that matches the analog channel dimension or width ACW” where the “the viewer will see the digitally derived image overlaid across the entire field of view of the analog derived image” and as in paragraph 0042 the digital image sensor 56b converts the enhanced image data to a digital form using ”a plurality of active or passive pixel sensor devices arranged in an array operating as optical pixels with CMOS circuity to convert the photons emitted from image intensifier 58 b to electrical signals, similar to the pixel array 40 shown in FIG. 3” where this array corresponds to the addressable array of output emitters responsible for displaying the digitized image and as in paragraphs 0030-0031 “Each pixel produces an electrical output signal in response to incident light or photons. The electrical signals are oftentimes read out, typically one row at a time, to form an image” and “the signals can be processed by processors 44, which may comprise analog-to-digital converters arranged on each column as the signals are read out by a column select unit 46. The electrical signals corresponding to each pixel output can be then placed on a bus 25” such that here the digital sensor 56b operates in this manner and provides a digital pixel array where these digital pixels are configured to be processed by “analog to digital converters” operating on each pixel); the digital image sensor is configured to read out a first row of the plurality of rows of sensor pixels and to bin image data corresponding to the first row in to a first data packet (see Smith, paragraphs 0030-0031 as explained in reference to the above teachings and explanation teaching “Each pixel produces an electrical output signal in response to incident light or photons. The electrical signals are oftentimes read out, typically one row at a time, to form an image” such that here the sensor is configured to ready a first row such as a first row of “one row at a time” of the multiple rows and the image data is functionally binned as it is read out by “one row at a time” bins and as they are read out in such bins this means the binned image data corresponds to the first row in a first data packet comprising the signals sent to the next stage for display given that the sending of the display data on the bus corresponds to display of the corresponding pixel on the corresponding digital array for display as in paragraphs 0039-0043 as explained above); and the digital display is configured to receive the first data packet and to fill a first row of display pixels with data comprising the first data packet, the first row of display pixels corresponding to the first row of sensor pixels (see Smith, paragraphs 0030-0031 as explained in reference to the above teachings and explanation teaching “Each pixel produces an electrical output signal in response to incident light or photons. The electrical signals are oftentimes read out, typically one row at a time, to form an image” such that here the sensor is configured to ready a first row such as a first row of “one row at a time” of the multiple rows and the image data is functionally binned as it is read out by “one row at a time” bins and as they are read out in such bins this means the binned image data corresponds to the first row in a first data packet comprising the signals sent to the next stage for display given that the sending of the display data on the bus corresponds to display of the corresponding pixel on the corresponding digital array for display as in paragraphs 0039-0043 as explained above). Thus Smith provides known techniques applicable to the base system of Dobbie.
Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to modify Dobbie with teachings of Smith to arrive at the claimed invention as doing so would be no more than application of a known technique applied to a base system ready for improvement, which would yield predictable results and result in an improved system. The predictable result of the combination would be that the digital image sensor and camera sensors of Dobbie would take the form of those in Smith to provide a digital camera sensor and technique for reading out such sensor data to a digital display such as the video display in Dobbie. Thus the modified Dobbie digital image sensor would output video according to the same principles while reading out and writing/filling data from the sensor to the output of the digital image sensor as in Smith. This would result in an improved system as the modified Dobbie system would then be capable of utilizing different arrangements for reading and processing image data according to techniques for improving reading/writing of digital data in arrays such as the suggested batching operation in Smith suggesting row-based readout, which further makes the system compatible with other arrangements such as pipelining and parallel operation for example.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dobbie as modified as applied to claim 9 above, and further in view of Guidash et al4 (“Guidash”).
Regarding claim 10, Dobbie as modified teaches all that is required as applied to claim 9 above but is silent to wherein: the digital image sensor is further configured to, while reading out the first row of sensor pixels, read out a second row of the plurality of rows of sensor pixels and to bin image data corresponding to the second row in to a second data packet; and the digital display is further configured to receive the second data packet and, while filling the first row of display pixels, to fill a second row of display pixels with data comprising the second data packet, the second row of display pixels corresponding to the second row of sensor pixels. Rather while Dobbie as modified as explained above would be compatible with such a pipelined concurrent filling approach, as rows are already read out binned into packets one row at a time to a display for output, there are no teachings as such. Thus Dobbie as modified stands as a base device upon which the claimed invention can be seen as an improvement through a reading and filling operation of data that would result in an increased throughput of data to data and could improve the latency response of the sensor to display pipeline.
In the same field of endeavor relating to low latency capturing and output of digital image data arranged in rows and read according to rows, Guidash teaches that it is known to while reading out the first row of sensor pixels, read out a second row of the plurality of rows of sensor pixels and to bin image data corresponding to the second row in to a second data packet; and the next stage is further configured to receive the second data packet and, while filling the first row of output pixels, to fill a second row of output pixels with data comprising the second data packet, the second row of display pixels corresponding to the second row of sensor pixels (see Guidash, paragraph 0201 teaching “readout of digital values (i.e., ADC results) stored within the digital output buffer 657 commences after completion of the large-signal ADC conversion (i.e., after any large-signal confirmed ADC results have been captured within the digital output buffer). In the embodiment shown, the digitized CDS results (i.e., ADC outputs or digital pixel values) may be shifted out of the digital output buffer for transmission to a memory IC and/or image-processing IC via a physical signaling interface (PHY) of the image sensor. In alternative embodiments, multiple digital pixel values may be output in parallel. Also, the digital line buffer may include separate “write-in” and “read-out” buffers (or an alternating buffer pair) to enable pixel data for a given pixel row to be output from the image sensor concurrently with storage of pixel data for the subsequent pixel row” such that here utilizing these “write-in” and “read-out” techniques it enables “pixel data for a given pixel row to be output from the image sensor concurrently with storage of pixel data for the subsequent pixel row”). Thus Guidash teaches a known technique applicable to the base system of Dobbie as modified.
Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to further modify Dobbie as modified as doing so would be no more than application of a known technique to a base system ready for improvement which would yield predictable results and result in an improved system. The predictable result of the combination would be that the data of the digital image sensor of Dobbie as already modified by Smith to be in row-by-row format would be read out and written according to Guidash’s technique. Thus the reading out of the sensor data would occur while concurrently writing or storing the next subsequent row to the next stage such that the data would be output to the display of Dobbie as modified by Smith and the display output stage would thus be filling a first row of display pixels while filling a second row of pixels allowing this concurrent reading and writing to an output stage. This would result in an improved system as the throughput to the display would be increased given the concurrent writing to the output display stage while not having to wait for other rows to finish reading, writing or filling.
Claim(s) 19-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Dobbie in view of Parker et al5 (“Parker”).
Regarding claim 19, Dobbie teaches all that is required as applied to claim 16 and further teaches wherein the external electronics processor is configured to operate, on the copy of the digital image, a trained machine learning model to generate the additional data (see Dobbie, column 11, lines 39-54 teaching “fundamental principle of image processing is to modify these values and produce an enhanced image for display. For each image processing function (contrast stretching, edge detection, etc.) there is a specific algorithm, or program, that must be performed. The control logic loads this program from a program RAM (not seen in FIG. 16) into the FPGA and uses the ALU to execute the algorithm. Essentially, the ALU numerically manipulates the pixel data in one of the frame buffers in accordance with the program logic. The specific image processing algorithms for different functions are well known to those skilled in the art”). Dobbie teaches the above but fails to specifically teach that the model used to generate the additional data is a trained machine learning model. Thus Dobbie stands as a base device upon which the claimed invention can be seen as an improvement through use of a trained machine learning model to operate on image data to generate additional data about the image, where benefits of such machine learning models are well known to those of ordinary skill in the art and allow for high speed processing of complex data to provide high speed complex outputs which would increase the ability of the system to enhance the images.
In the same field of endeavor relating to high speed processing of a video feed to add information to digital image information, Parker teaches that it is known to use an electronics processor to operate on digital image data, a trained machine learning model to generate additional data to add to the image data that can then be passed for display (see Parker, paragraph 0026 teaching “direct enhanced view optic (DEVO) according to embodiments of the present invention provides a user with enhanced target acquisition information, such as real time ballistic solutions, fused thermal imaging, extended zoom, and automatic target recognition” and “can include one or more of the following components and capabilities… a high performance, low power image processing and neural network system” and “real-time image processing for inputted video, such as high dynamic range, sensor fusion, contrast enhancement, multi-spectral, and low-light processing; displaying of geo-referenced augmented reality information; neural network automatic target recognition and highlighting (a box can be placed around people visible in the scope field of view or a “lock” on a user selectable object)” and as in paragraph 0055 “a graphical representation of a view 550 through a direct enhanced view optic with a holographic display system can include target tracking. In this embodiment, a target is identified and a bounding box 554 (e.g., 554 a-554 d) is positioned around the target. So long as the bound target is in view of the sensor, it can be tracked. A convolutional neural network can be implemented such that the light engine can detect desired shapes, such as people, faces, weapons, vehicles, etc., and highlight them for the user, for example by including an identifier such as “Car” as shown in FIG. 10” such that here a convolutional neural network is a trained machine learning model which has been trained to process the video feed and add information such as highlighting objects and detecting events such as objects coming in to view). Thus Parker provides know techniques applicable to the base system of Dobbie.
Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the invention to modify Dobbie to incorporate the use of the trained machine learning model as in Parker as doing so would be no more than application of a known technique to a base device ready for improvement and would yield predictable results and result in an improved system. The predictable result would be that the high performance and low power processor of Parker with a trained machine learning model could be used as the image processor as in Dobbie to additionally provide such information using that technique, with the objects and events being detected in the video stream of Dobbie’s system using Parker’s trained machine learning model to provide the additional information. This would result in an improved system as Dobbie’s system would be enabled to utilized high speed but low power machine learning processing and would allow for more complex processing and provision of more complex information about the captured scene and would allow the system to “provides a user with enhanced target acquisition information, such as real time ballistic solutions, fused thermal imaging, extended zoom, and automatic target recognition” and “reduce time to acquire targets, increase situational awareness and first shot probability of hit, reduce mental loading, reduce training requirements, and increase overall lethality”, as suggested by Parker (see Parker, paragraph 0026).
Regarding claim 20, Dobbie as modified teaches all that is required as applied to claim 19 above and further teaches wherein the machine learning model is trained to recognize one or more objects (see Dobbie as modified by Parker where as modified the trained machine learning model of Parker teaches to recognize one or more objects as in paragraph 0026 teaching “direct enhanced view optic (DEVO) according to embodiments of the present invention provides a user with enhanced target acquisition information, such as real time ballistic solutions, fused thermal imaging, extended zoom, and automatic target recognition” and “can include one or more of the following components and capabilities… a high performance, low power image processing and neural network system” and “real-time image processing for inputted video, such as high dynamic range, sensor fusion, contrast enhancement, multi-spectral, and low-light processing; displaying of geo-referenced augmented reality information; neural network automatic target recognition and highlighting (a box can be placed around people visible in the scope field of view or a “lock” on a user selectable object)” and as in paragraph 0055 “a graphical representation of a view 550 through a direct enhanced view optic with a holographic display system can include target tracking. In this embodiment, a target is identified and a bounding box 554 (e.g., 554 a-554 d) is positioned around the target. So long as the bound target is in view of the sensor, it can be tracked. A convolutional neural network can be implemented such that the light engine can detect desired shapes, such as people, faces, weapons, vehicles, etc., and highlight them for the user, for example by including an identifier such as “Car” as shown in FIG. 10” such that here a convolutional neural network is a trained machine learning model which has been trained to process the video feed and add information such as highlighting objects and detecting events such as objects coming in to view).
Regarding claim 21, Dobbie as modified teaches all that is required as applied to claim 19 above and further teaches wherein the machine learning model is trained to detect events (see Dobbie as modified by Parker where as modified the trained machine learning model of Parker teaches to recognize one or more objects as in paragraph 0026 teaching “direct enhanced view optic (DEVO) according to embodiments of the present invention provides a user with enhanced target acquisition information, such as real time ballistic solutions, fused thermal imaging, extended zoom, and automatic target recognition” and “can include one or more of the following components and capabilities… a high performance, low power image processing and neural network system” and “real-time image processing for inputted video, such as high dynamic range, sensor fusion, contrast enhancement, multi-spectral, and low-light processing; displaying of geo-referenced augmented reality information; neural network automatic target recognition and highlighting (a box can be placed around people visible in the scope field of view or a “lock” on a user selectable object)” and as in paragraph 0055 “a graphical representation of a view 550 through a direct enhanced view optic with a holographic display system can include target tracking. In this embodiment, a target is identified and a bounding box 554 (e.g., 554 a-554 d) is positioned around the target. So long as the bound target is in view of the sensor, it can be tracked. A convolutional neural network can be implemented such that the light engine can detect desired shapes, such as people, faces, weapons, vehicles, etc., and highlight them for the user, for example by including an identifier such as “Car” as shown in FIG. 10” such that here a convolutional neural network is a trained machine learning model which has been trained to process the video feed and add information such as highlighting objects and detecting events such as objects coming in to view).
Response to Arguments
Applicant's arguments filed 1/20/2026 have been fully considered but they are not persuasive.
Regarding Applicant’s arguments on page 7, of “REMARKS”, under “Claim objections”, Applicant argues with regard to amended claims 4, 5, 11, 14 and 15, that allowable limitations in the currently presented claims should lead to certain objections being withdrawn and that claims 11 and 15 are allowable. The Examiner agrees that claims 11 and 15 are allowable as they now have been rewritten to include all subject matter of the claims previously indicated to be allowable. However, claim 4 remains objected to as dependent upon a rejected base claims as explained below. Furthermore claim 5 is now rejected as explained above given the new scope of the claims. Furthermore note that claim 1 was not amended to include elements of claim 4 but was amended to included partial elements of claim 5, while broadening the claim in other respects, leading to the new rejections applied above.
Applicant’s arguments with respect to claim 1 on page 8, with respect to the new limitations allegedly not being taught by Chenderovitch are moot as the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The claims have now been amended such that the scope is reconsidered leading to the rejections of the claims using the prior art above. Thus Dobbie teaches the limitations argued missing from Chenderovitch as fully explained above.
Allowable Subject Matter
Claims 4 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 4, instant claim provides further limitations to those of claim 3 where it was specified that “wherein the pass-through electronics are configured to add information to the digital image data, wherein the information comprises one or more of color information, brightness information, calibration information, and augmented reality (AR) overlay information,” further requiring “wherein the pass-through electronics are configured to add information to the digital image data, wherein the information comprises one or more of color information, brightness information, calibration information, and augmented reality (AR) overlay information.” Thus it is required that the pass through electronics are now operating such that the digital display of claim 1 is now specifically configured to display YUV color data and the digital image sensor must be configured to generate a Y element of the YUV color data which the digital display is configured to display. Additionally “the color information” refers to the “color information” which is “information” which the pass-through electronics is configured to “add” per claim 3 and thus this also means the pass-through electronics are required to not add color information in the alternative but to be configured to add such color information using the pass-through electronics, which per claim 4 such color information that is added comprises a U element and a V element such that it must be supplied to the pass-through electronics in such a form. While night vision IITs output a single channel which could correspond to a Y channel in a YUV signal given that they simply produce a monochrome image, this would not necessarily mean that providing the Y element of YUV data in such format from a digital image sensor and digital display arrangement as required by the claim is known or obvious. Additionally, even as the IIT is no longer claimed in the parent claim, the external controller providing the additional information along with all of the limitations of claim 3 and 4 still require the novel and non-obvious formatting and processing of the Y, U and V elements as recited. Furthermore, color information from pass-through electronics so arranged and functioning as claimed in a U and V format would not correspond to such techniques as they would not be providing a Y element from a digital image sensor, so that pass-through electronics can add color information of a U and V component. The Examiner is unable to find any teaching or suggestion in the prior art which teaches or suggests the respective claim limitations when considered as a whole.
Claims 11-13 and 15 are allowed.
The following is an examiner’s statement of reasons for allowance: the prior art of record fails to teach or suggest the respective claim limitations when considered as a whole.
Regarding claim 11, the instant claim has been amended to incorporate all of the subject matter of previously presented claim 14, which was indicated as allowable in the Non-Final Rejection sent 10/23/2025, on page 25. Thus the claims are allowed for the reasons presented therein. Dependent claims 12 and 13 are allowable at least based on their dependence upon an allowable base claim.
Regarding claim 15, the instant claim has been amended to incorporate all of the subject matter of previously presented claim 15 including all of the limitations of previously presented claim 1 which were indicated as allowable in the Non-Final Rejection sent 10/23/2025, on pages 25-26. Thus the claims are allowed for the reasons presented therein.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SCOTT E SONNERS whose telephone number is (571)270-7504. The examiner can normally be reached Mon-Friday 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SCOTT E SONNERS/Examiner, Art Unit 2613
/XIAO M WU/Supervisory Patent Examiner, Art Unit 2613
1 US Patent No. 6560029
2 US PGPUB No. 20120274800
3 US PGPUB NO. 20210335566
4 20150229859
5 US PGPUB No. 20210262758
Prosecution Insights