DETAILED ACTION
This action is in response to the application filed 08/27/2024. Claims 1 – 12 are pending and have
been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 – 2 are rejected under 35 U.S.C. 103 as being unpatentable over Frodo et al. (JP Patent No. 2025516019, hereinafter “Frodo”) in view of Ashika (U.S. Pub. No. 2013/0021434, hereinafter “Ashika”) and Petrov (U.S. Patent No. 11,729,356).
Regarding Claim 1, Frodo teaches
A wireless multi-stream bidirectional video processing system (see Frodo Paragraph [0003], Video telephony comprises a remote communication system that uses computing devices to transmit audio and visual signals, allowing multiple people from different locations to participate in a conversation in real time) comprising:
a video input and output device receiving at least one camera video from at least one camera device and performing a first processing to the at least one camera video to become a video information (see Frodo Paragraph [0006], receive a video stream for the video conferencing session from a first client device; receive from the first client device visual effects information for one or more visual effects to apply to the video stream; apply the one or more visual effects to the video stream based on the received visual effects information to generate one or more modified video streams); and
a video processing device wirelessly communicating with the video input and output device to receive the video information (see Frodo Paragraph [0006], and transmit the one or more modified video streams to one or more other client devices participating in the video conferencing session),
Frodo does not expressively teach
performing a second processing to the video information to become a plurality of output videos, and transmitting the output videos to a host device,
wherein the host device outputs at least one display video to the video processing device, and
wherein the wireless multi-stream bidirectional video processing system processes the at least one display video to become a device video, and displays the device video on a display device connected to the wireless multi-stream bidirectional video processing system.
However, Ashika teaches
performing a second processing to the video information to become a plurality of output videos, and transmitting the output videos to a host device (see Ashika Paragraph [0109], According to various embodiments, there is provided: A method of transmitting a plurality of views from a video camera comprising: (a) capturing a scene in a wide-angle video camera and extracting at least two views of a possible plurality of views; (b) scaling the plurality of extracted views to independently and respectively specified sizes, where each view is a geometry corrected wide-angle view of a region of interest selectively extracted from one of a plurality of places within the wide-angle view, each independently scaled and each corresponding to an independently controlled virtual camera; (c) compositing at least a portion of the plurality of views into one or more views to form a composite video which includes multiple different views according to step (b) selectively superimposed, adjacent, or tiled, which can include selecting at least one region of interest from within the wide-angle view; (d) whereby a compositing of superimposed, adjacent, or tiled views can be accomplished by the camera, or by a base station using a plurality of composited views, wherein the base station supports a surveillance monitor able to display the at least two views simultaneously, superimposed, adjacent, or tiled, and individually scaled as selected, and selectively positioned in the framed view of the monitor),
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device (as taught in Frodo), with processing video data into multiple output videos to send to a host device (as taught in Ashika), the motivation being to provide an efficient, multi-angle view for simultaneous viewing of multiple video outputs (see Ashika Paragraphs [0039] and [0073]).
Frodo in view of Ashika do not expressively teach
wherein the host device outputs at least one display video to the video processing device, and
wherein the wireless multi-stream bidirectional video processing system processes the at least one display video to become a device video, and displays the device video on a display device connected to the wireless multi-stream bidirectional video processing system.
However, Petrov teaches
wherein the host device outputs at least one display video to the video processing device (see Petrov Column 7, lines 43 – 67 and Column 8, lines 1 – 9, Referring now to FIGS. 3A-3B, an example of method flow 300 for masking the video feedback loop in accordance with some embodiments is shown. At step 310 a first image associated with a display screen being shared is captured. For example, data associated with the screen being shared 205 is received by a processor, graphics unit, a memory component, etc., in order to be previewed in a preview window. The preview window is rendered by the processor and/or graphics unit on a display device and within the screen being shared. At step 312, the brightness of the first image that is captured is altered to form a second image, e.g., as displayed by window 210. For example, the brightness may be altered by the graphics unit and/or processor. It is appreciated that the brightness is altered in such a way that the alteration is substantially undetectable by human eye. In some embodiments, the brightness may be increased by a small amount that is predetermined. However, it is appreciated that in some embodiments the brightness may be decreased by a small amount. It is appreciated that while the alteration of the images is described with respect to altering the brightness, the example embodiments are not limited thereto and alteration of brightness should not be construed as limiting the scope of the embodiments. For example, in some embodiments altering the image may be by inserting a substantially unperceivable pixel or removing a pixel in such a way that is substantially undetectable by human eye. In other embodiments, the image may be altered by changing a color of one or more pixels in such a way that it is substantially undetectable by human eye. At step 314, the altered image is rendered in the preview window, e.g., window 210. For example, a processor or a graphics unit may render the altered image in the preview window. Thus, the shared screen may now include windows 205 and 210 forming a third image), and
wherein the wireless multi-stream bidirectional video processing system processes the at least one display video to become a device video, and displays the device video on a display device connected to the wireless multi-stream bidirectional video processing system (see Petrov Column 7, lines 43 – 67 and Column 8, lines 1 – 9, Referring now to FIGS. 3A-3B, an example of method flow 300 for masking the video feedback loop in accordance with some embodiments is shown. At step 310 a first image associated with a display screen being shared is captured. For example, data associated with the screen being shared 205 is received by a processor, graphics unit, a memory component, etc., in order to be previewed in a preview window. The preview window is rendered by the processor and/or graphics unit on a display device and within the screen being shared. At step 312, the brightness of the first image that is captured is altered to form a second image, e.g., as displayed by window 210. For example, the brightness may be altered by the graphics unit and/or processor. It is appreciated that the brightness is altered in such a way that the alteration is substantially undetectable by human eye. In some embodiments, the brightness may be increased by a small amount that is predetermined. However, it is appreciated that in some embodiments the brightness may be decreased by a small amount. It is appreciated that while the alteration of the images is described with respect to altering the brightness, the example embodiments are not limited thereto and alteration of brightness should not be construed as limiting the scope of the embodiments. For example, in some embodiments altering the image may be by inserting a substantially unperceivable pixel or removing a pixel in such a way that is substantially undetectable by human eye. In other embodiments, the image may be altered by changing a color of one or more pixels in such a way that it is substantially undetectable by human eye. At step 314, the altered image is rendered in the preview window, e.g., window 210. For example, a processor or a graphics unit may render the altered image in the preview window. Thus, the shared screen may now include windows 205 and 210 forming a third image).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device (as taught in Frodo), with processing video data into multiple output videos to send to a host device (as taught in Ashika), the motivation being to provide an efficient, multi-angle view for simultaneous viewing of multiple video outputs (see Ashika Paragraphs [0039] and [0073]).
It would have been further obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device (as taught in Frodo in view of Ashika), with a host device that sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Petrov), the motivation being to implement a two-way video system to enable more flexible use and reduce confusion for users (see Petrov Column 7, lines 43 – 67 and Column 8, lines 1 – 9, and Column 1, lines 65 - 67).
Regarding Claim 2, Frodo in view of Ashika and Petrov teaches
The wireless multi-stream bidirectional video processing system according to claim 1,
wherein the video processing device provides a plurality of endpoints for the host device according to a quantity of the output videos and the at least one display video, so as to transmit the output videos to the host device and receive the at least one display video from the host device (see Ashika Paragraph [0109], According to various embodiments, there is provided: A method of transmitting a plurality of views from a video camera comprising: (a) capturing a scene in a wide-angle video camera and extracting at least two views of a possible plurality of views; (b) scaling the plurality of extracted views to independently and respectively specified sizes, where each view is a geometry corrected wide-angle view of a region of interest selectively extracted from one of a plurality of places within the wide-angle view, each independently scaled and each corresponding to an independently controlled virtual camera; (c) compositing at least a portion of the plurality of views into one or more views to form a composite video which includes multiple different views according to step (b) selectively superimposed, adjacent, or tiled, which can include selecting at least one region of interest from within the wide-angle view; (d) whereby a compositing of superimposed, adjacent, or tiled views can be accomplished by the camera, or by a base station using a plurality of composited views, wherein the base station supports a surveillance monitor able to display the at least two views simultaneously, superimposed, adjacent, or tiled, and individually scaled as selected, and selectively positioned in the framed view of the monitor), and
wherein the host device generates the at least one display video according to the output videos (see Ashika Paragraph [0109], According to various embodiments, there is provided: A method of transmitting a plurality of views from a video camera comprising: (a) capturing a scene in a wide-angle video camera and extracting at least two views of a possible plurality of views; (b) scaling the plurality of extracted views to independently and respectively specified sizes, where each view is a geometry corrected wide-angle view of a region of interest selectively extracted from one of a plurality of places within the wide-angle view, each independently scaled and each corresponding to an independently controlled virtual camera; (c) compositing at least a portion of the plurality of views into one or more views to form a composite video which includes multiple different views according to step (b) selectively superimposed, adjacent, or tiled, which can include selecting at least one region of interest from within the wide-angle view; (d) whereby a compositing of superimposed, adjacent, or tiled views can be accomplished by the camera, or by a base station using a plurality of composited views, wherein the base station supports a surveillance monitor able to display the at least two views simultaneously, superimposed, adjacent, or tiled, and individually scaled as selected, and selectively positioned in the framed view of the monitor).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Frodo et al. (JP Patent No. 2025516019, hereinafter “Frodo”) in view of Ashika (U.S. Pub. No. 2013/0021434, hereinafter “Ashika”), Petrov (U.S. Patent No. 11,729,356) and Augustin et al. (U.S. Pub. No. 2004/0133708, hereinafter “Augustin”).
Regarding Claim 3, Frodo in view of Ashika and Petrov teaches
The wireless multi-stream bidirectional video processing system according to claim 1, wherein the video processing device comprises:
a wireless communication unit wirelessly receiving the video information from the video input and output device and transmitting a display information to the video input and output device (see Frodo Paragraph [0006], and transmit the one or more modified video streams to one or more other client devices participating in the video conferencing session);
an image processing unit performing the second processing to the video information to become the output videos (see Ashika Paragraph [0109], According to various embodiments, there is provided: A method of transmitting a plurality of views from a video camera comprising: (a) capturing a scene in a wide-angle video camera and extracting at least two views of a possible plurality of views; (b) scaling the plurality of extracted views to independently and respectively specified sizes, where each view is a geometry corrected wide-angle view of a region of interest selectively extracted from one of a plurality of places within the wide-angle view, each independently scaled and each corresponding to an independently controlled virtual camera; (c) compositing at least a portion of the plurality of views into one or more views to form a composite video which includes multiple different views according to step (b) selectively superimposed, adjacent, or tiled, which can include selecting at least one region of interest from within the wide-angle view; (d) whereby a compositing of superimposed, adjacent, or tiled views can be accomplished by the camera, or by a base station using a plurality of composited views, wherein the base station supports a surveillance monitor able to display the at least two views simultaneously, superimposed, adjacent, or tiled, and individually scaled as selected, and selectively positioned in the framed view of the monitor, and Figure 2, item 145 image processing circuitry);
a mode control unit adjusting the at least one display video received from the host device to become the display information (see Ashika Paragraph [0109], According to various embodiments, there is provided: A method of transmitting a plurality of views from a video camera comprising: (a) capturing a scene in a wide-angle video camera and extracting at least two views of a possible plurality of views; (b) scaling the plurality of extracted views to independently and respectively specified sizes, where each view is a geometry corrected wide-angle view of a region of interest selectively extracted from one of a plurality of places within the wide-angle view, each independently scaled and each corresponding to an independently controlled virtual camera; (c) compositing at least a portion of the plurality of views into one or more views to form a composite video which includes multiple different views according to step (b) selectively superimposed, adjacent, or tiled, which can include selecting at least one region of interest from within the wide-angle view; (d) whereby a compositing of superimposed, adjacent, or tiled views can be accomplished by the camera, or by a base station using a plurality of composited views, wherein the base station supports a surveillance monitor able to display the at least two views simultaneously, superimposed, adjacent, or tiled, and individually scaled as selected, and selectively positioned in the framed view of the monitor, in which the output is display-ready composition).
Frodo in view of Ashika and Petrov do not expressively teach
a USB control unit receiving the output videos, reporting an endpoint information to the host device according to the output videos, and receiving the at least one display video, so as to provide a plurality of endpoints for the host device according to the quantity of the output videos and the at least one display video;
a connection port unit receiving the output videos from the USB control unit to transmit the output videos to the host device, and receiving the at least one display video from the host device to transmit the at least one display video to the USB control unit;
However, Augustin teaches
a USB control unit receiving the output videos, reporting an endpoint information to the host device according to the output videos, and receiving the at least one display video, so as to provide a plurality of endpoints for the host device according to the quantity of the output videos and the at least one display video (see Augustin Paragraph [0016], FIG. 1 is a block diagram of a USB device virtual endpoint system useful for illustrating one embodiment of the invention. As shown in FIG. 1, system 10 includes a USB host controller 12 in communication with a USB device 14 through a USB bus 16. Device 14 is equipped with virtual endpoint capability and is in communication with local bus 18 of system 10. The local bus 18 connects to device 14 a central processing unit ("CPU") 20, a random access memory ("RAM") 22 and a programmable read only memory ("PROM") 24. Where system 10 comprises a USB host controller 12 and a peripheral device, the CPU 20, RAM 22 and PROM 24 would all form part of the peripheral as would device 14, Paragraph [0017], Virtual Endpoint hardware support in device 14 (also referred to herein as the USB device controller) enables firmware in PROM 24 to implement any number of USB device endpoints, Paragraph [0002], These functions may include printing, signal transmission through telephone lines by means of modems, video and audio signal transmission, telephone communication and broadband transmission, Paragraph [0006], When the host computer attempts to communicate with a USB peripheral device through a data pipe to support a particular function, the host computer will send a token containing information related to a logical endpoint for the data pipe. When the token is received by the USB device controller in the USB peripheral device, the USB device controller would match the logical endpoint information with that of the physical endpoints of the peripheral device, Paragraph [0004], Thus, if data is to be transferred in both directions between the host computer and the peripheral device for a particular function or application, two data pipes would be required: one for the host computer to send data to the peripheral device and the other for the peripheral device to send data to the host computer. Certain types of functions or applications may require more than one data pipe in the same direction);
a connection port unit receiving the output videos from the USB control unit to transmit the output videos to the host device, and receiving the at least one display video from the host device to transmit the at least one display video to the USB control unit (see Augustin Paragraph [0016], FIG. 1, system 10 includes a USB host controller 12 in communication with a USB device 14 through a USB bus 16, Paragraph [0020], When the host controller 12 sends an IN or OUT token to device 14 through bus 16, where the token refers to or contains a logical endpoint, the USB Device Controller (device 14) determines whether the packet matches a currently enabled Physical Endpoint Configuration. If it matches, then the USB Device Controller responds normally to the token, Paragraph [0017], Virtual Endpoint hardware support in device 14 (also referred to herein as the USB device controller) enables firmware in PROM 24 to implement any number of USB device endpoints, Paragraph [0002], These functions may include printing, signal transmission through telephone lines by means of modems, video and audio signal transmission, telephone communication and broadband transmission, Paragraph [0006], When the host computer attempts to communicate with a USB peripheral device through a data pipe to support a particular function, the host computer will send a token containing information related to a logical endpoint for the data pipe. When the token is received by the USB device controller in the USB peripheral device, the USB device controller would match the logical endpoint information with that of the physical endpoints of the peripheral device, Paragraph [0004], Thus, if data is to be transferred in both directions between the host computer and the peripheral device for a particular function or application, two data pipes would be required: one for the host computer to send data to the peripheral device and the other for the peripheral device to send data to the host computer. Certain types of functions or applications may require more than one data pipe in the same direction);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika and Petrov), with a USB control unit that manages multiple data streams by sending output data to host device and coordinating through connection ports to enable multi-endpoint communication (as taught in Augustin), the motivation being to provide an adequate amount of endpoints that are needed for supporting an increasing number of functions and applications desired by consumers in computer peripherals (see Augustin Paragraph [0008]).
Claims 4 – 5 are rejected under 35 U.S.C. 103 as being unpatentable over Frodo et al. (JP Patent No. 2025516019, hereinafter “Frodo”) in view of Ashika (U.S. Pub. No. 2013/0021434, hereinafter “Ashika”), Petrov (U.S. Patent No. 11,729,356), Augustin et al. (U.S. Pub. No. 2004/0133708, hereinafter “Augustin”) and Chatterjee et al. (U.S. Patent No. 12,276,715, hereinafter “Chatterjee).
Regarding Claim 4, Frodo in view of Ashika, Petrov and Augustin teach all the limitations of claim 3, but do not expressively teach
The wireless multi-stream bidirectional video processing system according to claim 3,
wherein the image processing unit of the video processing device includes a video processor, a neural network processor, a memory, and a storage unit, and wherein the video processor and the neural network processor use a program stored in the storage unit to perform the processing, and the memory is used to store the at least one video and the output videos,
wherein the processing is based on results of AI computing of the neural network processor, and
wherein the AI computing includes at least one of edge detection, image segmentation, voice direction detection, feature extraction, capturing body shapes, identifying persons, and arranging an image output based on the direction of human voices.
However, Chatterjee teaches
The wireless multi-stream bidirectional video processing system according to claim 3,
wherein the image processing unit of the video processing device includes a video processor, a neural network processor, a memory, and a storage unit, and wherein the video processor and the neural network processor use a program stored in the storage unit to perform the processing, and the memory is used to store the at least one video and the output videos (see Chatterjee Column 16, lines 63 – 67, and Column 17, lines 1 – 6, the reconstruction model may be a deep neural network with a plurality of hidden layers. Method 900 may be executed by a processor of an image processing system, such as the image processing system 202 of FIG. 2. Method 900 may be carried out according to instructions stored in non-transitory memory of the image processing system (e.g., in a training module such as the training module 212 of the image processing system 202 of FIG. 2) and executed by a processor of the image processing system (e.g., the processor 204 of image processing system 202 of FIG. 2)),
wherein the processing is based on results of AI computing of the neural network processor (see Chatterjee Column 16, lines 63 – 67, and Column 17, lines 1 – 6, the reconstruction model may be a deep neural network with a plurality of hidden layers. Method 900 may be executed by a processor of an image processing system, such as the image processing system 202 of FIG. 2. Method 900 may be carried out according to instructions stored in non-transitory memory of the image processing system (e.g., in a training module such as the training module 212 of the image processing system 202 of FIG. 2) and executed by a processor of the image processing system (e.g., the processor 204 of image processing system 202 of FIG. 2)), and
wherein the AI computing includes at least one of edge detection, image segmentation, voice direction detection, feature extraction, capturing body shapes, identifying persons, and arranging an image output based on the direction of human voices (see Chatterjee Column 7, lines 14 – 15, The model learns to detect image features, Column 1, lines 19 – 20, encode spatial information into the signal to generate an image).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika, Petrov and Augustin), with an image processing unit that uses a video processor and a neural network processor running stored programs to process and store video, generating output based on AI computations such as feature extraction (as taught in Chatterjee), the motivation being to improve processing quality and handle storage and output more efficiently (see Chatterjee Column 16, lines 63 – 67, and Column 17, lines 1 – 6).
Regarding Claim 5, Frodo in view of Ashika, Petrov, Augustin and Chatterjee teaches
The wireless multi-stream bidirectional video processing system according to claim 4, wherein the mode control unit includes:
a video conversion unit for adjusting the at least one display video (see Frodo Paragraph [0061], That is, the video processing server 300 is configured to perform a video transcoding operation using a graph of nodes. Transcoding refers to operations such as decompressing (e.g., decoding) a video stream, modifying the decompressed video stream (e.g., applying one or more visual effects, resizing the video stream, etc.), and recompressing (e.g., encoding) the video stream),
wherein the mode control unit (see Ashika Paragraph [0109], According to various embodiments, there is provided: A method of transmitting a plurality of views from a video camera comprising: (a) capturing a scene in a wide-angle video camera and extracting at least two views of a possible plurality of views; (b) scaling the plurality of extracted views to independently and respectively specified sizes, where each view is a geometry corrected wide-angle view of a region of interest selectively extracted from one of a plurality of places within the wide-angle view, each independently scaled and each corresponding to an independently controlled virtual camera; (c) compositing at least a portion of the plurality of views into one or more views to form a composite video which includes multiple different views according to step (b) selectively superimposed, adjacent, or tiled, which can include selecting at least one region of interest from within the wide-angle view; (d) whereby a compositing of superimposed, adjacent, or tiled views can be accomplished by the camera, or by a base station using a plurality of composited views, wherein the base station supports a surveillance monitor able to display the at least two views simultaneously, superimposed, adjacent, or tiled, and individually scaled as selected, and selectively positioned in the framed view of the monitor, in which the output is display-ready composition) receives the at least one display video from the USB control unit (see Augustin Paragraph [0016], FIG. 1, system 10 includes a USB host controller 12 in communication with a USB device 14 through a USB bus 16, Paragraph [0020], When the host controller 12 sends an IN or OUT token to device 14 through bus 16, where the token refers to or contains a logical endpoint, the USB Device Controller (device 14) determines whether the packet matches a currently enabled Physical Endpoint Configuration. If it matches, then the USB Device Controller responds normally to the token, Paragraph [0017], Virtual Endpoint hardware support in device 14 (also referred to herein as the USB device controller) enables firmware in PROM 24 to implement any number of USB device endpoints, Paragraph [0002], These functions may include printing, signal transmission through telephone lines by means of modems, video and audio signal transmission, telephone communication and broadband transmission, Paragraph [0006], When the host computer attempts to communicate with a USB peripheral device through a data pipe to support a particular function, the host computer will send a token containing information related to a logical endpoint for the data pipe. When the token is received by the USB device controller in the USB peripheral device, the USB device controller would match the logical endpoint information with that of the physical endpoints of the peripheral device, Paragraph [0004], Thus, if data is to be transferred in both directions between the host computer and the peripheral device for a particular function or application, two data pipes would be required: one for the host computer to send data to the peripheral device and the other for the peripheral device to send data to the host computer. Certain types of functions or applications may require more than one data pipe in the same direction), and
wherein the adjustment includes adjusting a format of the at least one display video and compressing the at least one display video after the format adjusting into the at least one display information, so as to facilitate wireless transmission (see Frodo Paragraph [0061], That is, the video processing server 300 is configured to perform a video transcoding operation using a graph of nodes. Transcoding refers to operations such as decompressing (e.g., decoding) a video stream, modifying the decompressed video stream (e.g., applying one or more visual effects, resizing the video stream, etc.), and recompressing (e.g., encoding) the video stream).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Frodo et al. (JP Patent No. 2025516019, hereinafter “Frodo”) in view of Ashika (U.S. Pub. No. 2013/0021434, hereinafter “Ashika”), Petrov (U.S. Patent No. 11,729,356), Augustin et al. (U.S. Pub. No. 2004/0133708, hereinafter “Augustin”), Chatterjee et al. (U.S. Patent No. 12,276,715, hereinafter “Chatterjee), Ansari et al. (U.S. Pub. No. 2023/0057851, hereinafter “Ansari”) and Zhou et al. (U.S. Pub. No. 2024/0013432, hereinafter “Zhou”).
Regarding Claim 6, Frodo in view of Ashika, Petrov, Augustin and Chatterjee teaches
The wireless multi-stream bidirectional video processing system according to claim 5,
wherein, before the mode control unit adjusts the at least one display video (see Ashika Paragraph [0109], According to various embodiments, there is provided: A method of transmitting a plurality of views from a video camera comprising: (a) capturing a scene in a wide-angle video camera and extracting at least two views of a possible plurality of views; (b) scaling the plurality of extracted views to independently and respectively specified sizes, where each view is a geometry corrected wide-angle view of a region of interest selectively extracted from one of a plurality of places within the wide-angle view, each independently scaled and each corresponding to an independently controlled virtual camera; (c) compositing at least a portion of the plurality of views into one or more views to form a composite video which includes multiple different views according to step (b) selectively superimposed, adjacent, or tiled, which can include selecting at least one region of interest from within the wide-angle view; (d) whereby a compositing of superimposed, adjacent, or tiled views can be accomplished by the camera, or by a base station using a plurality of composited views, wherein the base station supports a surveillance monitor able to display the at least two views simultaneously, superimposed, adjacent, or tiled, and individually scaled as selected, and selectively positioned in the framed view of the monitor, in which the output is display-ready composition), the image processing unit of the video processing device performs a third processing to the at least one display video (see Ashika Paragraph [0109], According to various embodiments, there is provided: A method of transmitting a plurality of views from a video camera comprising: (a) capturing a scene in a wide-angle video camera and extracting at least two views of a possible plurality of views; (b) scaling the plurality of extracted views to independently and respectively specified sizes, where each view is a geometry corrected wide-angle view of a region of interest selectively extracted from one of a plurality of places within the wide-angle view, each independently scaled and each corresponding to an independently controlled virtual camera; (c) compositing at least a portion of the plurality of views into one or more views to form a composite video which includes multiple different views according to step (b) selectively superimposed, adjacent, or tiled, which can include selecting at least one region of interest from within the wide-angle view; (d) whereby a compositing of superimposed, adjacent, or tiled views can be accomplished by the camera, or by a base station using a plurality of composited views, wherein the base station supports a surveillance monitor able to display the at least two views simultaneously, superimposed, adjacent, or tiled, and individually scaled as selected, and selectively positioned in the framed view of the monitor, and Figure 2, item 145 image processing circuitry),
wherein the third processing of the at least one display video includes at least one video display mode, the image processing unit of the video processing device performs the third processing to the at least one display video according to the video display mode that is selected (see Frodo Paragraph [0033], According to an exemplary embodiment, a client device determines whether to process video for a videoconferencing session in a first mode or a second mode, in which there are multiple modes, and Paragraph [0089], the visual effects mode selector 134),
wherein the video display mode includes picture zoom-in and zoom-out (see Frodo Paragraph [0055], in an embodiment, when the zoom function is enabled in the video conferencing application at the client device, the video processing server may be configured to apply visual effects to the video stream in the first mode. For example, a magnification stream (i.e., a raw video stream modified with zoom at the client) may be sent from the client device to the video processing server, and visual effects may be applied to the magnification stream. For example, a raw video stream may be sent from the client device to the video processing server, and the video processing server may apply both the magnification and visual effects to the video stream)
Frodo in view of Ashika, Petrov, Augustin and Chatterjee do not expressively teach
wherein the video display mode includes picture-in-picture, side-by-side picture, picture cropping, picture overlapping, and original picture.
However, Ansari teaches
wherein the video display mode includes picture-in-picture (see Ansari Paragraph [0066], Digital media device 35 may support picture-in-picture (PIP) functionality, and may further support the ability for a user to configure the size and location of the PIP window within a display (e.g., a display presented on television display 32)), side-by-side picture (see Ansari Paragraph [0066], digital media device 35 may support the display of two windows of video side-by-side), picture overlapping (see Ansari Paragraph [0066], Digital media device 35 may be configured to receive a selection (e.g., from remote control 550) as to an audio signal or stream to be played by a television or audio/video system for PIP, POP, PIG, side-by-side, or composite video images (e.g., overlays or insert displays), or any suitable combination thereof)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika, Petrov, Augustin and Chatterjee), with a video display mode including picture-in-picture, side-by-side picture, and picture overlapping (as taught in Ansari), the motivation being to provide a dynamic interface that allows users to view multiple videos at the same time in a single screen (see Ansari Paragraph [0066]).
Frodo in view of Ashika, Petrov, Augustin, Chatterjee and Ansari do not expressively teach
wherein the video display mode includes picture cropping and original picture.
However, Zhou teaches
wherein the video display mode includes picture cropping (see Zhou Paragraph [0378], The upper left sub-window in FIG. 20b is the 2D display window, which is used to represent an image acquired by a camera at a current viewing angle. The image may be an original image acquired or an image obtained through image transformation. If the camera is the primary camera, the upper left sub-window may display the original image acquired by the primary camera, or an image obtained by cropping (such as cropping to remove the black edge and retain the central region) the original image acquired by the primary camera) and original picture (see Zhou Paragraph [0378], The upper left sub-window in FIG. 20b is the 2D display window, which is used to represent an image acquired by a camera at a current viewing angle. The image may be an original image acquired or an image obtained through image transformation. If the camera is the primary camera, the upper left sub-window may display the original image acquired by the primary camera, or an image obtained by cropping (such as cropping to remove the black edge and retain the central region) the original image acquired by the primary camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika, Petrov, Augustin and Chatterjee), with a video display mode including picture-in-picture, side-by-side picture, and picture overlapping (as taught in Ansari), the motivation being to provide a dynamic interface that allows users to view multiple videos at the same time in a single screen (see Ansari Paragraph [0066]).
It would have been further obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display and the system has multiple video display modes (as taught in Frodo in view of Ashika, Petrov, Augustin, Chatterjee and Ansari), with a video display mode includes picture cropping and original picture (as taught in Zhou), the motivation being to provide different video display modes for users to ensure the video is output efficiently (see Zhou Paragraph [0378]).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Frodo et al. (JP Patent No. 2025516019, hereinafter “Frodo”) in view of Ashika (U.S. Pub. No. 2013/0021434, hereinafter “Ashika”), Petrov (U.S. Patent No. 11,729,356), Augustin et al. (U.S. Pub. No. 2004/0133708, hereinafter “Augustin”), Chatterjee et al. (U.S. Patent No. 12,276,715, hereinafter “Chatterjee), Ansari et al. (U.S. Pub. No. 2023/0057851, hereinafter “Ansari”), Zhou et al. (U.S. Pub. No. 2024/0013432, hereinafter “Zhou”) and McCaughan et al. (U.S. Pub. No. 2018/0108121, hereinafter “McCaughan”).
Regarding Claim 7, Frodo in view of Ashika, Petrov, Augustin, Chatterjee, Ansari and Zhou teaches
The wireless multi-stream bidirectional video processing system according to claim 6,
wherein the display device is connected to the video processing device (see Frodo Paragraph [0006], and transmit the one or more modified video streams to one or more other client devices participating in the video conferencing session, and Paragraph [0024], FIG. 5 illustrates an exemplary display of a video conferencing session on a client device),
wherein the third processing further includes processing the at least one display video to become the device video (see Petrov Column 7, lines 43 – 67 and Column 8, lines 1 – 9, Referring now to FIGS. 3A-3B, an example of method flow 300 for masking the video feedback loop in accordance with some embodiments is shown. At step 310 a first image associated with a display screen being shared is captured. For example, data associated with the screen being shared 205 is received by a processor, graphics unit, a memory component, etc., in order to be previewed in a preview window. The preview window is rendered by the processor and/or graphics unit on a display device and within the screen being shared. At step 312, the brightness of the first image that is captured is altered to form a second image, e.g., as displayed by window 210. For example, the brightness may be altered by the graphics unit and/or processor. It is appreciated that the brightness is altered in such a way that the alteration is substantially undetectable by human eye. In some embodiments, the brightness may be increased by a small amount that is predetermined. However, it is appreciated that in some embodiments the brightness may be decreased by a small amount. It is appreciated that while the alteration of the images is described with respect to altering the brightness, the example embodiments are not limited thereto and alteration of brightness should not be construed as limiting the scope of the embodiments. For example, in some embodiments altering the image may be by inserting a substantially unperceivable pixel or removing a pixel in such a way that is substantially undetectable by human eye. In other embodiments, the image may be altered by changing a color of one or more pixels in such a way that it is substantially undetectable by human eye. At step 314, the altered image is rendered in the preview window, e.g., window 210. For example, a processor or a graphics unit may render the altered image in the preview window. Thus, the shared screen may now include windows 205 and 210 forming a third image), and
wherein the video processing device further includes:
a video input controller for controlling a video received from another host device (see Augustin Paragraph [0006], When the host computer attempts to communicate with a USB peripheral device through a data pipe to support a particular function, the host computer will send a token containing information related to a logical endpoint for the data pipe. When the token is received by the USB device controller in the USB peripheral device, the USB device controller would match the logical endpoint information with that of the physical endpoints of the peripheral device, Paragraph [0004], Thus, if data is to be transferred in both directions between the host computer and the peripheral device for a particular function or application, two data pipes would be required: one for the host computer to send data to the peripheral device and the other for the peripheral device to send data to the host computer. Certain types of functions or applications may require more than one data pipe in the same direction);
a video output controller for controlling the device video output to the display device (see Petrov Column 7, lines 43 – 67 and Column 8, lines 1 – 9, Referring now to FIGS. 3A-3B, an example of method flow 300 for masking the video feedback loop in accordance with some embodiments is shown. At step 310 a first image associated with a display screen being shared is captured. For example, data associated with the screen being shared 205 is received by a processor, graphics unit, a memory component, etc., in order to be previewed in a preview window. The preview window is rendered by the processor and/or graphics unit on a display device and within the screen being shared. At step 312, the brightness of the first image that is captured is altered to form a second image, e.g., as displayed by window 210. For example, the brightness may be altered by the graphics unit and/or processor. It is appreciated that the brightness is altered in such a way that the alteration is substantially undetectable by human eye. In some embodiments, the brightness may be increased by a small amount that is predetermined. However, it is appreciated that in some embodiments the brightness may be decreased by a small amount. It is appreciated that while the alteration of the images is described with respect to altering the brightness, the example embodiments are not limited thereto and alteration of brightness should not be construed as limiting the scope of the embodiments. For example, in some embodiments altering the image may be by inserting a substantially unperceivable pixel or removing a pixel in such a way that is substantially undetectable by human eye. In other embodiments, the image may be altered by changing a color of one or more pixels in such a way that it is substantially undetectable by human eye. At step 314, the altered image is rendered in the preview window, e.g., window 210. For example, a processor or a graphics unit may render the altered image in the preview window. Thus, the shared screen may now include windows 205 and 210 forming a third image);
Frodo in view of Ashika, Petrov, Augustin, Chatterjee, Ansari and Zhou do not expressively teach
a video input connection port for receiving the video from the another host device to transmit the video to the video input controller; and
a video output connection port for receiving the device video from the image processing unit to transmit the device video to the display device.
However, McCaughan teaches
a video input connection port for receiving the video from the another host device to transmit the video to the video input controller (see McCaughan Paragraph [0108], Data capture, processing and sharing system 200 comprises a computer system 202. Various components of the computer system 202 are connected to a bus 212 of computer system 202. In this example, an input/output port 204 is connected to both of bus 212 and camera 104, respectively. According to the present example, the computer system 202 further comprises a processor 206, a communication port 214, a program memory with image processing software, operating system and communication software 210, a mass storage 205, a working memory 208 and a printer 216, Paragraph [0112], Generally, camera 104 captures an image, which is then processed using computer system 202 or a computer system integral to camera 104, respectively. The processed and updated image may then be transmitted to external devices via communication port 214); and
a video output connection port for receiving the device video from the image processing unit to transmit the device video to the display device (see McCaughan Paragraph [0108], Data capture, processing and sharing system 200 comprises a computer system 202. Various components of the computer system 202 are connected to a bus 212 of computer system 202. In this example, an input/output port 204 is connected to both of bus 212 and camera 104, respectively. According to the present example, the computer system 202 further comprises a processor 206, a communication port 214, a program memory with image processing software, operating system and communication software 210, a mass storage 205, a working memory 208 and a printer 216, Paragraph [0112], Generally, camera 104 captures an image, which is then processed using computer system 202 or a computer system integral to camera 104, respectively. The processed and updated image may then be transmitted to external devices via communication port 214).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display and the system has multiple video display modes (as taught in Frodo in view of Ashika, Petrov, Augustin, Chatterjee, Ansari and Zhou), with input and output connection ports to receive and transmit video data (as taught in McCaughan), the motivation being input/output ports provide increased reliability, consistency and predictability for data transmission (see McCaughan Paragraph [0108]).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Frodo et al. (JP Patent No. 2025516019, hereinafter “Frodo”) in view of Ashika (U.S. Pub. No. 2013/0021434, hereinafter “Ashika”), Petrov (U.S. Patent No. 11,729,356), Ansari et al. (U.S. Pub. No. 2023/0057851, hereinafter “Ansari”) and Zhou et al. (U.S. Pub. No. 2024/0013432, hereinafter “Zhou”).
Regarding Claim 8, Frodo in view of Ashika and Petrov teaches
The wireless multi-stream bidirectional video processing system according to claim 1,
wherein the second processing includes at least one video display mode, the image processing unit of the video processing device performs the second processing to the at least one video according to the video display mode that is selected (see Frodo Paragraph [0033], According to an exemplary embodiment, a client device determines whether to process video for a videoconferencing session in a first mode or a second mode, in which there are multiple modes, and Paragraph [0089], the visual effects mode selector 134), and
wherein the video display mode includes picture zoom-in and zoom-out (see Frodo Paragraph [0055], in an embodiment, when the zoom function is enabled in the video conferencing application at the client device, the video processing server may be configured to apply visual effects to the video stream in the first mode. For example, a magnification stream (i.e., a raw video stream modified with zoom at the client) may be sent from the client device to the video processing server, and visual effects may be applied to the magnification stream. For example, a raw video stream may be sent from the client device to the video processing server, and the video processing server may apply both the magnification and visual effects to the video stream)
Frodo in view of Ashika and Petrov do not expressively teach
wherein the video display mode includes picture-in-picture, side-by-side picture, picture cropping, picture overlapping, and original picture.
However, Ansari teaches
wherein the video display mode includes picture-in-picture (see Ansari Paragraph [0066], Digital media device 35 may support picture-in-picture (PIP) functionality, and may further support the ability for a user to configure the size and location of the PIP window within a display (e.g., a display presented on television display 32)), side-by-side picture (see Ansari Paragraph [0066], digital media device 35 may support the display of two windows of video side-by-side), picture overlapping (see Ansari Paragraph [0066], Digital media device 35 may be configured to receive a selection (e.g., from remote control 550) as to an audio signal or stream to be played by a television or audio/video system for PIP, POP, PIG, side-by-side, or composite video images (e.g., overlays or insert displays), or any suitable combination thereof)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika and Petrov), with a video display mode including picture-in-picture, side-by-side picture, and picture overlapping (as taught in Ansari), the motivation being to provide a dynamic interface that allows users to view multiple videos at the same time in a single screen (see Ansari Paragraph [0066]).
Frodo in view of Ashika, Petrov and Ansari do not expressively teach
wherein the video display mode includes picture cropping and original picture.
However, Zhou teaches
wherein the video display mode includes picture cropping (see Zhou Paragraph [0378], The upper left sub-window in FIG. 20b is the 2D display window, which is used to represent an image acquired by a camera at a current viewing angle. The image may be an original image acquired or an image obtained through image transformation. If the camera is the primary camera, the upper left sub-window may display the original image acquired by the primary camera, or an image obtained by cropping (such as cropping to remove the black edge and retain the central region) the original image acquired by the primary camera) and original picture (see Zhou Paragraph [0378], The upper left sub-window in FIG. 20b is the 2D display window, which is used to represent an image acquired by a camera at a current viewing angle. The image may be an original image acquired or an image obtained through image transformation. If the camera is the primary camera, the upper left sub-window may display the original image acquired by the primary camera, or an image obtained by cropping (such as cropping to remove the black edge and retain the central region) the original image acquired by the primary camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika and Petrov), with a video display mode including picture-in-picture, side-by-side picture, and picture overlapping (as taught in Ansari), the motivation being to provide a dynamic interface that allows users to view multiple videos at the same time in a single screen (see Ansari Paragraph [0066]).
It would have been further obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display and the system has multiple video display modes (as taught in Frodo in view of Ashika, Petrov and Ansari), with a video display mode includes picture cropping and original picture (as taught in Zhou), the motivation being to provide different video display modes for users to ensure the video is output efficiently (see Zhou Paragraph [0378]).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Frodo et al. (JP Patent No. 2025516019, hereinafter “Frodo”) in view of Ashika (U.S. Pub. No. 2013/0021434, hereinafter “Ashika”), Petrov (U.S. Patent No. 11,729,356), Ansari et al. (U.S. Pub. No. 2023/0057851, hereinafter “Ansari”), Zhou et al. (U.S. Pub. No. 2024/0013432, hereinafter “Zhou”) and McCaughan et al. (U.S. Pub. No. 2018/0108121, hereinafter “McCaughan”).
Regarding Claim 9, Frodo in view of Ashika, Petrov, Ansari and Zhou teach all the limitations of claim 8, but do not expressively teach
The wireless multi-stream bidirectional video processing system according to claim 8, wherein the second processing further includes:
a format processing for converting a format of the received at least one video into a format that complies with a USB video class, so as to allow the connection port unit to transmit the output videos; and
a resolution processing for converting a resolution of the received at least one video into a resolution consistent with the host device.
However, McCaughan teaches
The wireless multi-stream bidirectional video processing system according to claim 8, wherein the second processing further includes:
a format processing for converting a format of the received at least one video into a format that complies with a USB video class, so as to allow the connection port unit to transmit the output videos (see McCaughan Paragraph [0039], In one embodiment of the method (and below-described system) the writing/drawing data is output in a form which simulates or emulates data from a camera such as a web-cam. In this way the output data may be provided to a video conferencing system, Skype™ system or the like without the need to modify the system to which it is retrofitted. In embodiments the writing/drawing data may be arranged to conform to a standard corresponding to a camera, data format standard, for example a UVC (USB video class) standard, Paragraph [0108], Data capture, processing and sharing system 200 comprises a computer system 202. Various components of the computer system 202 are connected to a bus 212 of computer system 202. In this example, an input/output port 204 is connected to both of bus 212 and camera 104, respectively. According to the present example, the computer system 202 further comprises a processor 206, a communication port 214, a program memory with image processing software, operating system and communication software 210, a mass storage 205, a working memory 208 and a printer 216, Paragraph [0112], Generally, camera 104 captures an image, which is then processed using computer system 202 or a computer system integral to camera 104, respectively. The processed and updated image may then be transmitted to external devices via communication port 214); and
a resolution processing for converting a resolution of the received at least one video into a resolution consistent with the host device (see McCaughan Paragraph [0054], Optionally the resolution of the system may be optimized to suit the board by providing a number of simple mechanical pre-set positions which match the size of common boards).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display and the system has multiple video display modes (as taught in Frodo in view of Ashika, Petrov, Ansari and Zhou), with converting video into a USB video class-compliant format and adjusting the video resolution to match the connected device (as taught in McCaughan), the motivation being to ensure video is recognized and supported by standard USB drivers and prevents display issues when transmitting video data (see McCaughan Paragraphs [0039], [0108], [0112] and [0054]).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Frodo et al. (JP Patent No. 2025516019, hereinafter “Frodo”) in view of Ashika (U.S. Pub. No. 2013/0021434, hereinafter “Ashika”), Petrov (U.S. Patent No. 11,729,356), Augustin et al. (U.S. Pub. No. 2004/0133708, hereinafter “Augustin”) and McCaughan et al. (U.S. Pub. No. 2018/0108121, hereinafter “McCaughan”).
Regarding Claim 10, Frodo in view of Ashika and Petrov teaches
The wireless multi-stream bidirectional video processing system according to claim 1,
wherein the display device is connected to the video input and output device (see Frodo Paragraph [0006], receive a video stream for the video conferencing session from a first client device; receive from the first client device visual effects information for one or more visual effects to apply to the video stream; apply the one or more visual effects to the video stream based on the received visual effects information to generate one or more modified video streams and transmit the one or more modified video streams to one or more other client devices participating in the video conferencing session, and Paragraph [0024], FIG. 5 illustrates an exemplary display of a video conferencing session on a client device), and
wherein the video input and output device includes:
a wireless communication unit for wirelessly transmitting the video information to the video processing device and receiving the display information from the video processing device (see Frodo Paragraph [0006], and transmit the one or more modified video streams to one or more other client devices participating in the video conferencing session, Paragraph [0057], For example, wireless communication between elements of exemplary embodiments may be performed via wireless LAN, Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), Ultra Wideband (UWB), Infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), radio frequency (RF) signals, etc.);
an image processing unit for performing the first processing to the at least one camera video to become the video information (see Ashika Paragraph [0109], According to various embodiments, there is provided: A method of transmitting a plurality of views from a video camera comprising: (a) capturing a scene in a wide-angle video camera and extracting at least two views of a possible plurality of views; (b) scaling the plurality of extracted views to independently and respectively specified sizes, where each view is a geometry corrected wide-angle view of a region of interest selectively extracted from one of a plurality of places within the wide-angle view, each independently scaled and each corresponding to an independently controlled virtual camera; (c) compositing at least a portion of the plurality of views into one or more views to form a composite video which includes multiple different views according to step (b) selectively superimposed, adjacent, or tiled, which can include selecting at least one region of interest from within the wide-angle view; (d) whereby a compositing of superimposed, adjacent, or tiled views can be accomplished by the camera, or by a base station using a plurality of composited views, wherein the base station supports a surveillance monitor able to display the at least two views simultaneously, superimposed, adjacent, or tiled, and individually scaled as selected, and selectively positioned in the framed view of the monitor, and Figure 2, item 145 image processing circuitry), and performing a fourth processing to the display information to become the device video (see Petrov Column 7, lines 43 – 67 and Column 8, lines 1 – 9, Referring now to FIGS. 3A-3B, an example of method flow 300 for masking the video feedback loop in accordance with some embodiments is shown. At step 310 a first image associated with a display screen being shared is captured. For example, data associated with the screen being shared 205 is received by a processor, graphics unit, a memory component, etc., in order to be previewed in a preview window. The preview window is rendered by the processor and/or graphics unit on a display device and within the screen being shared. At step 312, the brightness of the first image that is captured is altered to form a second image, e.g., as displayed by window 210. For example, the brightness may be altered by the graphics unit and/or processor. It is appreciated that the brightness is altered in such a way that the alteration is substantially undetectable by human eye. In some embodiments, the brightness may be increased by a small amount that is predetermined. However, it is appreciated that in some embodiments the brightness may be decreased by a small amount. It is appreciated that while the alteration of the images is described with respect to altering the brightness, the example embodiments are not limited thereto and alteration of brightness should not be construed as limiting the scope of the embodiments. For example, in some embodiments altering the image may be by inserting a substantially unperceivable pixel or removing a pixel in such a way that is substantially undetectable by human eye. In other embodiments, the image may be altered by changing a color of one or more pixels in such a way that it is substantially undetectable by human eye. At step 314, the altered image is rendered in the preview window, e.g., window 210. For example, a processor or a graphics unit may render the altered image in the preview window. Thus, the shared screen may now include windows 205 and 210 forming a third image);
a video output controller for controlling the device video that is output to the display device (see Petrov Column 7, lines 43 – 67 and Column 8, lines 1 – 9, Referring now to FIGS. 3A-3B, an example of method flow 300 for masking the video feedback loop in accordance with some embodiments is shown. At step 310 a first image associated with a display screen being shared is captured. For example, data associated with the screen being shared 205 is received by a processor, graphics unit, a memory component, etc., in order to be previewed in a preview window. The preview window is rendered by the processor and/or graphics unit on a display device and within the screen being shared. At step 312, the brightness of the first image that is captured is altered to form a second image, e.g., as displayed by window 210. For example, the brightness may be altered by the graphics unit and/or processor. It is appreciated that the brightness is altered in such a way that the alteration is substantially undetectable by human eye. In some embodiments, the brightness may be increased by a small amount that is predetermined. However, it is appreciated that in some embodiments the brightness may be decreased by a small amount. It is appreciated that while the alteration of the images is described with respect to altering the brightness, the example embodiments are not limited thereto and alteration of brightness should not be construed as limiting the scope of the embodiments. For example, in some embodiments altering the image may be by inserting a substantially unperceivable pixel or removing a pixel in such a way that is substantially undetectable by human eye. In other embodiments, the image may be altered by changing a color of one or more pixels in such a way that it is substantially undetectable by human eye. At step 314, the altered image is rendered in the preview window, e.g., window 210. For example, a processor or a graphics unit may render the altered image in the preview window. Thus, the shared screen may now include windows 205 and 210 forming a third image);
Frodo in view of Ashika and Petrov do not expressively teach
a video input controller for controlling the received at least one camera video;
a video input connection port for receiving the at least one camera video from the at least one camera device to transmit the at least one camera video to the video input controller; and
a video output connection port for receiving the device video from the video output controller to transmit the device video to the display device.
However, Augustin teaches
a video input controller for controlling the received at least one camera video (see Augustin Paragraph [0006], When the host computer attempts to communicate with a USB peripheral device through a data pipe to support a particular function, the host computer will send a token containing information related to a logical endpoint for the data pipe. When the token is received by the USB device controller in the USB peripheral device, the USB device controller would match the logical endpoint information with that of the physical endpoints of the peripheral device, Paragraph [0004], Thus, if data is to be transferred in both directions between the host computer and the peripheral device for a particular function or application, two data pipes would be required: one for the host computer to send data to the peripheral device and the other for the peripheral device to send data to the host computer. Certain types of functions or applications may require more than one data pipe in the same direction);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika and Petrov), with a video input controller for controlling the received at least one camera video (as taught in Augustin), the motivation being to manage and organize video prior to processing (see Augustin Paragraph [0006]).
Frodo in view of Ashika, Petrov and Augustin do not expressively teach
a video input connection port for receiving the at least one camera video from the at least one camera device to transmit the at least one camera video to the video input controller; and
a video output connection port for receiving the device video from the video output controller to transmit the device video to the display device.
However, McCaughan teaches
a video input connection port for receiving the at least one camera video from the at least one camera device to transmit the at least one camera video to the video input controller (see McCaughan Paragraph [0108], Data capture, processing and sharing system 200 comprises a computer system 202. Various components of the computer system 202 are connected to a bus 212 of computer system 202. In this example, an input/output port 204 is connected to both of bus 212 and camera 104, respectively. According to the present example, the computer system 202 further comprises a processor 206, a communication port 214, a program memory with image processing software, operating system and communication software 210, a mass storage 205, a working memory 208 and a printer 216, Paragraph [0112], Generally, camera 104 captures an image, which is then processed using computer system 202 or a computer system integral to camera 104, respectively. The processed and updated image may then be transmitted to external devices via communication port 214); and
a video output connection port for receiving the device video from the video output controller to transmit the device video to the display device (see McCaughan Paragraph [0108], Data capture, processing and sharing system 200 comprises a computer system 202. Various components of the computer system 202 are connected to a bus 212 of computer system 202. In this example, an input/output port 204 is connected to both of bus 212 and camera 104, respectively. According to the present example, the computer system 202 further comprises a processor 206, a communication port 214, a program memory with image processing software, operating system and communication software 210, a mass storage 205, a working memory 208 and a printer 216, Paragraph [0112], Generally, camera 104 captures an image, which is then processed using computer system 202 or a computer system integral to camera 104, respectively. The processed and updated image may then be transmitted to external devices via communication port 214).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika and Petrov), with a video input controller for controlling the received at least one camera video (as taught in Augustin), the motivation being to manage and organize video prior to processing (see Augustin Paragraph [0006]).
It would have been further obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika, Petrov and Augustin), with input and output connection ports to receive and transmit video data (as taught in McCaughan), the motivation being input/output ports provide increased reliability, consistency and predictability for data transmission (see McCaughan Paragraph [0108]).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Frodo et al. (JP Patent No. 2025516019, hereinafter “Frodo”) in view of Ashika (U.S. Pub. No. 2013/0021434, hereinafter “Ashika”), Petrov (U.S. Patent No. 11,729,356), Augustin et al. (U.S. Pub. No. 2004/0133708, hereinafter “Augustin”), McCaughan et al. (U.S. Pub. No. 2018/0108121, hereinafter “McCaughan”) and Chatterjee et al. (U.S. Patent No. 12,276,715, hereinafter “Chatterjee).
Regarding Claim 11, Frodo in view of Ashika, Petrov, Augustin and McCaughan teach all the limitations of claim 10, but do not expressively teach
The wireless multi-stream bidirectional video processing system according to claim 10,
wherein the image processing unit of the video input and output device includes a video processor, a neural network processor, a memory, and a storage unit,
wherein the video processor and the neural network processor use a program stored in the storage unit to perform the processing, and the memory is used to store the at least one camera video and the device videos,
wherein the processing is based on results of AI computing of the neural network processor, and
wherein the AI computing includes at least one of edge detection, image segmentation, voice direction detection, feature extraction, capturing body shapes, identifying persons, and arranging an image output based on the direction of human voices.
However, Chatterjee teaches
The wireless multi-stream bidirectional video processing system according to claim 10,
wherein the image processing unit of the video input and output device includes a video processor, a neural network processor, a memory, and a storage unit (see Chatterjee Column 16, lines 63 – 67, and Column 17, lines 1 – 6, the reconstruction model may be a deep neural network with a plurality of hidden layers. Method 900 may be executed by a processor of an image processing system, such as the image processing system 202 of FIG. 2. Method 900 may be carried out according to instructions stored in non-transitory memory of the image processing system (e.g., in a training module such as the training module 212 of the image processing system 202 of FIG. 2) and executed by a processor of the image processing system (e.g., the processor 204 of image processing system 202 of FIG. 2), Column 3, line 8, a data acquisition unit 24),
wherein the video processor and the neural network processor use a program stored in the storage unit to perform the processing, and the memory is used to store the at least one camera video and the device videos (see Chatterjee Column 16, lines 63 – 67, and Column 17, lines 1 – 6, the reconstruction model may be a deep neural network with a plurality of hidden layers. Method 900 may be executed by a processor of an image processing system, such as the image processing system 202 of FIG. 2. Method 900 may be carried out according to instructions stored in non-transitory memory of the image processing system (e.g., in a training module such as the training module 212 of the image processing system 202 of FIG. 2) and executed by a processor of the image processing system (e.g., the processor 204 of image processing system 202 of FIG. 2)),
wherein the processing is based on results of AI computing of the neural network processor (see Chatterjee Column 16, lines 63 – 67, and Column 17, lines 1 – 6, the reconstruction model may be a deep neural network with a plurality of hidden layers. Method 900 may be executed by a processor of an image processing system, such as the image processing system 202 of FIG. 2. Method 900 may be carried out according to instructions stored in non-transitory memory of the image processing system (e.g., in a training module such as the training module 212 of the image processing system 202 of FIG. 2) and executed by a processor of the image processing system (e.g., the processor 204 of image processing system 202 of FIG. 2)), and
wherein the AI computing includes at least one of edge detection, image segmentation, voice direction detection, feature extraction, capturing body shapes, identifying persons, and arranging an image output based on the direction of human voices (see Chatterjee Column 7, lines 14 – 15, The model learns to detect image features, Column 1, lines 19 – 20, encode spatial information into the signal to generate an image).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika, Petrov, Augustin and McCaughan), with an image processing unit that uses a video processor and a neural network processor running stored programs to process and store video, generating output based on AI computations such as feature extraction (as taught in Chatterjee), the motivation being to improve processing quality and handle storage and output more efficiently (see Chatterjee Column 16, lines 63 – 67, and Column 17, lines 1 – 6).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Frodo et al. (JP Patent No. 2025516019, hereinafter “Frodo”) in view of Ashika (U.S. Pub. No. 2013/0021434, hereinafter “Ashika”), Petrov (U.S. Patent No. 11,729,356), Augustin et al. (U.S. Pub. No. 2004/0133708, hereinafter “Augustin”), McCaughan et al. (U.S. Pub. No. 2018/0108121, hereinafter “McCaughan”), Ansari et al. (U.S. Pub. No. 2023/0057851, hereinafter “Ansari”) and Zhou et al. (U.S. Pub. No. 2024/0013432, hereinafter “Zhou”).
Regarding Claim 12, Frodo in view of Ashika, Petrov, Augustin and McCaughan teach
The wireless multi-stream bidirectional video processing system according to claim 10,
wherein the first processing and the fourth processing include at least one video display mode, the image processing unit of the video input and output device performs the first processing and the fourth processing according to the video display mode that is selected (see Frodo Paragraph [0033], According to an exemplary embodiment, a client device determines whether to process video for a videoconferencing session in a first mode or a second mode, in which there are multiple modes, and Paragraph [0089], the visual effects mode selector 134), and
wherein the video display mode includes picture zoom-in and zoom-out (see Frodo Paragraph [0055], in an embodiment, when the zoom function is enabled in the video conferencing application at the client device, the video processing server may be configured to apply visual effects to the video stream in the first mode. For example, a magnification stream (i.e., a raw video stream modified with zoom at the client) may be sent from the client device to the video processing server, and visual effects may be applied to the magnification stream. For example, a raw video stream may be sent from the client device to the video processing server, and the video processing server may apply both the magnification and visual effects to the video stream)
Frodo in view of Ashika, Petrov, Augustin and McCaughan do not expressively teach
wherein the video display mode includes picture-in-picture, side-by-side picture, picture cropping, picture overlapping, and original picture.
However, Ansari teaches
wherein the video display mode includes picture-in-picture (see Ansari Paragraph [0066], Digital media device 35 may support picture-in-picture (PIP) functionality, and may further support the ability for a user to configure the size and location of the PIP window within a display (e.g., a display presented on television display 32)), side-by-side picture (see Ansari Paragraph [0066], digital media device 35 may support the display of two windows of video side-by-side), picture overlapping (see Ansari Paragraph [0066], Digital media device 35 may be configured to receive a selection (e.g., from remote control 550) as to an audio signal or stream to be played by a television or audio/video system for PIP, POP, PIG, side-by-side, or composite video images (e.g., overlays or insert displays), or any suitable combination thereof)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika, Petrov, Augustin and McCaughan), with a video display mode including picture-in-picture, side-by-side picture, and picture overlapping (as taught in Ansari), the motivation being to provide a dynamic interface that allows users to view multiple videos at the same time in a single screen (see Ansari Paragraph [0066]).
Frodo in view of Ashika, Petrov, Augustin, McCaughan and Ansari do not expressively teach
wherein the video display mode includes picture cropping and original picture.
However, Zhou teaches
wherein the video display mode includes picture cropping (see Zhou Paragraph [0378], The upper left sub-window in FIG. 20b is the 2D display window, which is used to represent an image acquired by a camera at a current viewing angle. The image may be an original image acquired or an image obtained through image transformation. If the camera is the primary camera, the upper left sub-window may display the original image acquired by the primary camera, or an image obtained by cropping (such as cropping to remove the black edge and retain the central region) the original image acquired by the primary camera) and original picture (see Zhou Paragraph [0378], The upper left sub-window in FIG. 20b is the 2D display window, which is used to represent an image acquired by a camera at a current viewing angle. The image may be an original image acquired or an image obtained through image transformation. If the camera is the primary camera, the upper left sub-window may display the original image acquired by the primary camera, or an image obtained by cropping (such as cropping to remove the black edge and retain the central region) the original image acquired by the primary camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display (as taught in Frodo in view of Ashika, Petrov, Augustin and McCaughan), with a video display mode including picture-in-picture, side-by-side picture, and picture overlapping (as taught in Ansari), the motivation being to provide a dynamic interface that allows users to view multiple videos at the same time in a single screen (see Ansari Paragraph [0066]).
It would have been further obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of a system in which at least one camera captures video, which is processed by a device and wirelessly transmitted to another device, and performing a second processing to process video data into multiple output videos to send to a host device, in which the host device sends a display video to a processing device, which converts it into device video and outputs to a connected display and the system has multiple video display modes (as taught in Frodo in view of Ashika, Petrov, Augustin, McCaughan and Ansari), with a video display mode includes picture cropping and original picture (as taught in Zhou), the motivation being to provide different video display modes for users to ensure the video is output efficiently (see Zhou Paragraph [0378]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of References Cited for a listing of analogous art.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARISSA A JONES whose telephone number is (703)756-1677. The examiner can normally be reached Telework M-F 6:30 AM - 4:00 PM CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at 5712727503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CARISSA A JONES/ Examiner, Art Unit 2691
/DUC NGUYEN/ Supervisory Patent Examiner, Art Unit 2691
