DETAILED ACTION
This action is in response to the communication filed 01/06/2026:
Claims 1 – 20 are pending
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claims 1 - 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Response to Amendment
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4, 7, 8, 11, 14, 15, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Holzer et al. (U.S. Publication No. 2018/0227601, hereinafter " Holzer ") in view of Powell (U.S. Pub. No. 2024/0112315).
Regarding Claim 1, Holzer teaches
A method (see Holzer Paragraph [0063], method), comprising:
receiving, at a server, an image from a video feed of a device, wherein the server is remote from the device that generates the video feed (see Holzer Paragraph [0006], transmitting a first video frame in a raw video stream from the client device to a server via a network);
identifying, using a server, a modification for the image (see Holzer Paragraph [0037] If the selected frame meets the designated criterion, then information about the selected frame is transmitted to the server at 208. According to various embodiments, a variety of information may be transmitted to the server. In one example, some or all of the image data associated with the frame may be transmitted. For instance, the entire frame may be transmitted. Alternately, the frame may be compressed or down sampled to reduce bandwidth usage. In a second example, IMU information such as gyroscopic data, compass data, or accelerometer data may be transmitted. This IMU information may provide data about the position, velocity, acceleration, direction, rotation, or other such characteristics of the device around the time that the frame was captured. In a third example, GPS information may be transmitted. In some implementations, the specific information transmitted to the server may depend on the type of processing being performed at the server and/or the type of filter being applied at the client device, and Paragraph [0046], Then, at 306 the server performs filter processing operations for the video stream. The specific filter processing operations performed may depend in large part on the particular type of filter being applied to the video stream. Some examples of processing operations running on the server may include, but are not limited to: detection, segmentation, and pose estimation. Such methods may be applied to objects that include, but are not limited to: humans, animals, vehicles, inanimate objects, and plants. Other examples of methods running on the server may include, but are not limited to: depth estimation, scene reconstruction, scene decomposition, and semantic labeling, in which the server conducts the improvement detection by detection, segmentation, etc. and the modification is a specific filter);
transmitting, by the server, the modification to a device, the device configured to generate a corrected image based on the modification (see Holzer Paragraph [0038] Next, a determination is made at 210 as to whether a new filter processing message has been received from the server. As shown in FIG. 1, the server sends messages that include information for applying filters to frames, Paragraph [0041], If instead a new filter processing message has been received, then at 214 a filter is applied based on both the locally available data and the data provided by the server. According to various embodiments, new information received from the server may be combined with the information propagated from frame to frame. To accomplish this goal, various approaches may be used. In one example, old information may be replaced with new information received from the server. In a second example, old information may continue to be used without alteration, for instance if the new information is close enough to the old information. In a third example, old information may be combined with new information in a weighted fashion, for instance based on relative confidence values associated with server results and propagation results, and Paragraph [0047], at 308, the filter processing information is transmitted to the client device. According to various embodiments, the filter processing operations may generate a wide range of information for transmission. For example, the filter processing operations may generate location information that identifies locations of high-level features such as faces or skeleton components on image data sent from the client device. As another example, the filter processing operations may include or identify virtual elements to overlay on top of the video stream at the client device. For instance, the filter processing operation may identify the video stream as including footage of a running dog and then indicate as one filter possibility a cape that could be overlain on the moving image of the dog to generate a visual effect of a “super dog.” As yet another example, the filter processing operations may include semantic elements such as labels for recognized objects or words generated by applying optical character recognition to text in the video stream. Thus, the specific information transmitted to the client device at operation 308 may depend in part upon the characteristics of the filter processing operations performed at the server, in which the device receives the instructions from the server and performs the actual image correction);
receiving an additional image from the video feed, wherein the additional image is not transmitted to the server for identifying another modification by the server (see Holzer Paragraph [0051], With reference to FIG. 6, shown is an example of a procedure for propagating information across successive frames in a video stream during live filtering. In the procedure 600, information associated with a first frame in a video stream is transmitted to a server for processing at 602. For instance, the second request message 173 shown in FIG. 6 may be sent from the client device to the server. Then, at 604, filter results are propagated for one or more intervening frames in the video stream, such as the frames 156, 157, and 158. According to various embodiments, information can be directly propagated to or between any other frames, for instance any frames between 153 and 159, in which the intervening frames are interpreted as additional images, Paragraph [0053] For example, the client device may be configured to identify features associated with the frames such as points or areas. A point may correspond, for instance, with an elbow, a table corner, a nose, or some other such physical object. An area may correspond, for instance, with a shirt, a table surface, a face, or some other such physical area of interest. The client device may be configured to identify such features in each frame. Then, the client device may construct a correspondence between successive frames. For instance, one or more points or areas may be located at similar locations across successive frames. Further, motion across successive frames may be tracked by detecting trajectories of points or areas across successive frames, and Paragraph [0054], high-level features identified by the server may be tracked by tracking lower-level features capable of being identified by the client device. For example, the server may indicate that a particular constellation of points or an area of color represents a face, and that a speech bubble is to be located next to the face. Then, the client device may track the constellation of points or area of color over successive frames and move the speech bubble as necessary to hold its position relative to the face);
generating, at the device and prior to transmission of the additional image, a corrected additional image by applying the modification that was previously received from the server for the image to the additional image (see Holzer Paragraph [0051], With reference to FIG. 6, shown is an example of a procedure for propagating information across successive frames in a video stream during live filtering. In the procedure 600, information associated with a first frame in a video stream is transmitted to a server for processing at 602. For instance, the second request message 173 shown in FIG. 6 may be sent from the client device to the server. Then, at 604, filter results are propagated for one or more intervening frames in the video stream, such as the frames 156, 157, and 158. According to various embodiments, information can be directly propagated to or between any other frames, for instance any frames between 153 and 159, in which the intervening frames are interpreted as additional images, Paragraph [0053] For example, the client device may be configured to identify features associated with the frames such as points or areas. A point may correspond, for instance, with an elbow, a table corner, a nose, or some other such physical object. An area may correspond, for instance, with a shirt, a table surface, a face, or some other such physical area of interest. The client device may be configured to identify such features in each frame. Then, the client device may construct a correspondence between successive frames. For instance, one or more points or areas may be located at similar locations across successive frames. Further, motion across successive frames may be tracked by detecting trajectories of points or areas across successive frames, and Paragraph [0054], high-level features identified by the server may be tracked by tracking lower-level features capable of being identified by the client device. For example, the server may indicate that a particular constellation of points or an area of color represents a face, and that a speech bubble is to be located next to the face. Then, the client device may track the constellation of points or area of color over successive frames and move the speech bubble as necessary to hold its position relative to the face); and
transmitting the corrected additional image from the device to a remote device for display (see Holzer Paragraph [0043], providing the filtered frame for presentation may involve displaying the filtered frame as part of the video stream on a display screen).
Holzer does not expressively teach
Conference room device that is connected to a video conference
an improvement detection engine at the server
an image correction engine that resides at the conference room device
receiving, by the server and from the conference room device, the corrected image from the image correction engine;
transmitting the corrected additional image from the conference room device to a remote device for display via the server.
However, Powell teaches
Conference room device that is connected to a video conference (see Powell Paragraph [0048], In another more particular example, video data source 106 can include an application configured to generate video stream data (e.g., a teleconferencing application with video streaming capabilities, a digital effect application, and/or a distortion correction application being executed by computing device 102, server 104, and/or any other suitable computing device) and Paragraph [0044], Computing device 102 may include a communication system 112)
an engine at the server (see Powell Paragraph [0045], Server 104 may include a communication system 112, a digital effect engine or component 114, and/or a distortion correction engine or component 116. In some examples, server 104 can execute at least a portion of the digital effect component 114 to apply a digital effect, such as a pan, tilt, and/or zoom effect, to a video stream. The digital effect may simulate a change in elevation or a change in azimuth of a lens optical axis, without physical moving an optical axis of the lens. Further, in some examples, server 104 can execute at least a portion of the distortion correction component 116 to correct a geometric distortion to one or more objects within a video stream (e.g., corresponding to the video stream data 110). The geometric distortion may be corrected by applying distortion correction techniques (e.g., dewarping) that may incorporate, for example, analytical projections or meshes)
an engine that resides at the conference room device (see Powell Paragraph [0044], Computing device 102 may include a communication system 112, a digital effect engine or component 114, and/or a distortion correction engine or component 116. In some examples, computing device 102 can execute at least a portion of the digital effect component 114 to apply a digital effect, such as a pan, tilt, and/or zoom effect, to a video stream. The digital effect may simulate a change in elevation or a change in azimuth of a lens optical axis, without physical moving an optical axis of the lens. Further, in some examples, computing device 102 can execute at least a portion of the distortion correction component 116 to correct a geometric distortion to one or more objects within a video stream (e.g., corresponding to the video stream data 110). The geometric distortion may be corrected by applying distortion correction techniques (e.g., dewarping) that may incorporate, for example, analytical projections or meshes)
receiving, by the server and from the conference room device, the corrected image from the engine (see Powell Paragraph [0044], Computing device 102 may include a communication system 112, a digital effect engine or component 114, and/or a distortion correction engine or component 116. In some examples, computing device 102 can execute at least a portion of the digital effect component 114 to apply a digital effect, such as a pan, tilt, and/or zoom effect, to a video stream. The digital effect may simulate a change in elevation or a change in azimuth of a lens optical axis, without physical moving an optical axis of the lens. Further, in some examples, computing device 102 can execute at least a portion of the distortion correction component 116 to correct a geometric distortion to one or more objects within a video stream (e.g., corresponding to the video stream data 110). The geometric distortion may be corrected by applying distortion correction techniques (e.g., dewarping) that may incorporate, for example, analytical projections or meshes, Figure 1, server includes communication system, Paragraph [0138], the server device 1202 may provide data to and from a client computing device such as a personal computer 1204, a tablet computing device 1206 and/or a mobile computing device 1208 (e.g., a smart phone) through a network 1215, Paragraph [0006], applying the digital effect, and outputting the modified stream of images);
transmitting the corrected additional image from the conference room device to a remote device for display via the server (see Powell Figure 1, server includes communication system, Figure 8, item 814, outputting the modified stream of images and item 816, displaying the modified stream of images, in which additional corrected images are created from the loop of process 800, Paragraph [0138], the server device 1202 may provide data to and from a client computing device such as a personal computer 1204, a tablet computing device 1206 and/or a mobile computing device 1208 (e.g., a smart phone) through a network 1215, Paragraph [0006], applying the digital effect, and outputting the modified stream of images, and Paragraph [0112], At operation 816, the modified stream of images is displayed. The modified stream of images may be displayed, for example, on a display screen of a computing device, such as computing device 102 described earlier herein with respect to FIG. 1. Additionally, or alternatively, the modified stream of images may be otherwise displayed using components that may be local to a device at which one or more of the above operations were performed, or remote from a device at which one or more of the above operations were performed, in which aspects of method 800 may be performed by a device and/or a server (Paragraph [0085])).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a client-server system that identifies a modification for an image within a video stream, applies the modification, then applies the same modification to additional images within the video stream (as taught in Holzer), with a client-server video conference system that implements an engine at the server and at the client device (as taught in Powell), the motivation being to balance performance and consistency by implementing distinct engines within the server and device to separate and balance actions when modifying images within a video conference (see Powell Paragraph [0044] and [0045]).
Regarding Claim 4, Holzer in view of Powell teaches
The method of claim 1, comprising:
transmitting the corrected additional image to the remote device for display (see Holzer Paragraph [0043], the filtered frame may be transmitted to a separate device for presentation, such as an augmented reality or virtual reality device in communication with the client device).
Regarding Claim 7, Holzer in view of Powell teaches
The method of claim 1, wherein the modification comprises an instruction for the image correction engine (see Holzer Paragraph [0026], The server processes the video frame to produce filtering information and then transmits a filter processing message to the client device that indicates how to apply a filter to the video frame. The client device then applies the filtering information to create a filtered video stream, as applied to engine in Powell).
Regarding Claims 8, 11, and 14, they are rejected similarly as Claims 1, 4, and 7, respectively. The non-transitory computer readable medium can be found in Holzer (Paragraph [0006], computer readable media).
Regarding Claims 15 and 18, they are rejected similarly as Claims 1 and 4, respectively. The apparatus, memory, processor to execute instructions stored in memory can be found in Holzer (Paragraph [0063], apparatus, memory, processor to execute instructions stored in memory).
Claims 2, 6, 9, 13, 16 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Holzer et al. (U.S. Publication No. 2018/0227601, hereinafter " Holzer ") in view of Powell (U.S. Pub. No. 2024/0112315) and Parampottil et al. (U.S. Patent No. 11659137, hereinafter "Parampottil").
Regarding Claim 2, Holzer in view of Powell does not expressively teach
The method of claim 1, wherein the modification comprises at least one of straightening the image, modifying a brightness level of the image, or placing a digital annotation to obscure a part of the image.
However, Parampottil teaches
The method of claim 1, wherein the modification comprises at least one of straightening the image, modifying a brightness level of the image, or placing a digital annotation to obscure a part of the image (see Parampottil FIG. 5B, the system is configured to blur static objects, so that object 415 is obscured 502 and not displayed).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a client-server method to identify improvements in an image and correcting accordingly for display in a video conference and applying modification to subsequent images (as taught in Holzer in view of Powell), with placing a digital annotation to obscure a part of the image as an image modification (as taught in Parampottil), the motivation being to obscure an extraneous item from view in a user's display to avoid unwanted parties from observing said extraneous items (see Parampottil Column 1, lines 17 – 25).
Regarding Claim 6, Holzer in view of Powell and Parampottil teaches
The method of claim 1, comprising:
transmitting, to an address associated with an administrator of the conference room device, a message comprising a representation of the modification (see Parampottil Column 12, lines 26 – 27, alerts and notifications may be sent when blurring is enabled, triggered, and/or applied to a video device and Column 5, line 60, requiring permission from a facility administrator).
Regarding Claims 9 and 13, they are rejected similarly as Claims 2 and 6, respectively. The non-transitory computer readable medium can be found in Holzer (Paragraph [0006], computer readable media).
Regarding Claims 16 and 20, they are rejected similarly as Claims 2 and 6, respectively. The apparatus, memory, processor to execute instructions stored in memory can be found in Holzer (Paragraph [0063], apparatus, memory, processor to execute instructions stored in memory).
Claims 3, 10, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Holzer et al. (U.S. Publication No. 2018/0227601, hereinafter " Holzer ") in view of Powell (U.S. Pub. No. 2024/0112315) and Choi et al. (K.R. Publication No. 20220102016, hereinafter "Choi").
Regarding Claim 3, Holzer in view of Powell does not expressively teach
The method of claim 1, wherein at least one of the improvement detection engine or the image correction engine comprises an artificial neural network.
However Choi teaches
The method of claim 1, wherein at least one of the improvement detection engine or the image correction engine comprises an artificial neural network (see Choi Paragraph [0006], artificial intelligence-based image processing technique to perform a correction function for image).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching a client-server method to identify improvements in an image and correcting accordingly for display in a video conference and applying modification to subsequent images (as taught in Holzer in view of Powell), with detecting or performing an image correction using an artificial neural network (as taught in Choi), the motivation being to provide the user the ability to easily and accurately recognize the correction result(s) according to the recommendation to obtain a desired level of improved image (see Choi Paragraph [0006]).
Regarding Claim 10, it is rejected similarly as Claim 3. The non-transitory computer readable medium can be found in Holzer (Paragraph [0006], computer readable media).
Regarding Claim 17, it is rejected similarly as Claim 3. The apparatus, memory, processor to execute instructions stored in memory can be found in Holzer (Paragraph [0063], apparatus, memory, processor to execute instructions stored in memory).
Claims 5, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Holzer et al. (U.S. Publication No. 2018/0227601, hereinafter " Holzer ") in view of Powell (U.S. Pub. No. 2024/0112315) and Shayne et al. (U.S. Publication No. 20230133750, hereinafter "Shayne").
Regarding Claim 5, Holzer in view of Powell does not expressively teach
The method of claim 1, comprising:
transmitting, to the conference room device, a prompt for approval of the corrected image, wherein transmitting the corrected image to the remote device is in response to receiving the approval.
However, Shayne teaches
The method of claim 1, comprising:
transmitting, to the conference room device, a prompt for approval of the corrected image, wherein transmitting the corrected image to the remote device is in response to receiving the approval (see Shayne Paragraph [0081], control unit may be programmed to request permission from a user, e.g., conferencer, before adjusting a device (such as modification in Holzer)).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a client-server method to identify improvements in an image and correcting accordingly for display in a video conference and applying modification to subsequent images (as taught in Holzer in view of Powell), with a prompt for approval of the image correction prior to display in the video conference (as taught in Shayne), the motivation being to allow the user to check the modification prior to transmitting to video conference in case of an error (see Shayne Paragraph [0081]).
Regarding Claim 12, it is rejected similarly as Claim 5, respectively. The non-transitory computer readable medium can be found in Holzer (Paragraph [0006], computer readable media).
Regarding Claim 19, it is rejected similarly as Claim 5, respectively. The apparatus, memory, processor to execute instructions stored in memory can be found in Holzer (Paragraph [0063], apparatus, memory, processor to execute instructions stored in memory).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of References Cited for a listing of analogous art.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARISSA A JONES whose telephone number is (703)756-1677. The examiner can normally be reached Telework M-F 6:30 AM - 4:00 PM CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached on 5712727503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CARISSA A JONES/Examiner, Art Unit 2691
/DUC NGUYEN/Supervisory Patent Examiner, Art Unit 2691