Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 35 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the enablement requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention Claim 35 claimed: “continuously capturing images with the first camera while the first viewing screen is not temporarily blacked out, and discarding those images captured with the first camera while the first viewing screen is not temporarily blacked out; and providing a sequence of the captured images over a network to at least a second viewing screen of a second participant.” How the discarded images captured with the first camera while the first viewing screen is not temporarily blacked out being provided over a network to a second viewing screen of a second participant? Which captured images are referring to? The image captured while the first screening is intermittently blacked out or the images captured while the first screening is not intermittently blacked out? It is noted that “capturing an image” is not the same as “capturing images”.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 9, 12-15, 17-18, and 29-31 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang (9,270,933).
Consider claim 1. Jiang teaches a method comprising: during a videoconference, with a first camera (140, fig. 1a-1c), capturing an image of a first participant reflected from a first viewing screen collocated with the first participant and the first camera, while the first viewing screen is intermittently blacked out (e.g., dark screen period 506, col. 5, line 60 to col. 6, line 28, fig. 5a-c) by causing the first viewing screen to display in a full screen mode (e.g., the entire display screen was darkened out); and providing a sequence of the captured images over a network to at least a second viewing screen of a second participant (col. 5, line 60 to col. 6, line 28, fig. 5a-c).
Jiang does not mention the first viewing screen is intermittently blacked out by causing the first viewing screen to display in a full screen mode to prevent emission of extraneous light during the intermittent black out. However, it would have been obvious to one skilled in the art that darkening a display screen would obviously prevent emission of extraneous light.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Jiang to solve the lacking of eye contacts problem during video conversation, because each of the communicators commonly looks at the video screen and not at the video camera, preventing the communicator from having direct eye-to-eye contact to the other remote viewers, as the video cameras are usually positioned away from the screen.
Consider claims 2 and 19. Jiang further teaches refraining from capturing images with the first camera while the first viewing screen is not temporarily blacked out (e.g., normal display period 505, during which the display device shows intended display content. The local user sees normally intended display screen image 510 which may include a sub-window 520 of the remote user's image, as shown in FIG. 5B).
Consider claim 12. Jiang further teaches the first viewing screen (120, fig 1a-c), coupled to the at least one processor (see claim 10 of Jiang), wherein the first viewing screen does not have a non-reflective coating (e.g., conventional glossy surface, col. 3, lines 44-54).
Consider claim 13. Jiang further teaches the at least one processor includes at least a processor of a desktop computer (e.g., either the laptop or desktop shown in fig. 1c and 2), further comprising: the first camera (140), wherein the first camera is an external web camera and is coupled to the at least one processor (see fig. 1c); and the first viewing screen (120, fig. 1c), coupled to the at least one processor (see claim 10 of Jiang); wherein the desktop computer includes a network interface coupled to the network (the computer 110 in fig 1c inherently comprises a network interface to perform video communication).
Consider claim 14. Jiang further teaches adjusting the first camera to point to a midpoint of the first viewing screen (fig. 8, col. 6, lines 42-54).
Consider claim 15. Jiang further teaches: tracking gaze of the first participant; and adjusting the first camera to point to a location on the first viewing screen corresponding to the gaze of the first participant (i.e., There can be various modes of camera tracking for eye contact. In one embodiment, the camera position and orientation are automatically adjusted to track eye gazing spot on the screen. For example, as illustrated in FIG. 4A, if the eye gazing spot is at location 420 of a display 410, the position and orientation of the camera 430 are adjusted to be in line with the reflection path 435 at spot 420 of the eye image; col. 5, lines 30-36).
Consider claim 17. Jiang teaches a non-transitory computer readable medium comprising computer executable instructions which when executed by a computer cause the computer (see claim 10 of Jiang: computer, software and hardware) to perform a method comprising: during a videoconference, with a first camera (140, fig. 1a-1c), capturing an image of a first participant reflected from a first viewing screen collocated with the first participant and the first camera, while the first viewing screen is intermittently blacked out (e.g., dark screen period 506, col. 5, line 60 to col. 6, line 28, fig. 5a-c) by causing the first viewing screen to display in a full screen mode (e.g., the entire display screen was darkened out); and providing a sequence of the captured images over a network to at least a second viewing screen of a second participant (col. 5, line 60 to col. 6, line 28, fig. 5a-c).
Jiang does not mention the first viewing screen is intermittently blacked out by causing the first viewing screen to display in a full screen mode to prevent emission of extraneous light during the intermittent black out. However, it would have been obvious to one skilled in the art that darkening a display screen would obviously prevent emission of extraneous light.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Jiang to solve the lacking of eye contacts problem during video conversation, because each of the communicators commonly looks at the video screen and not at the video camera, preventing the communicator from having direct eye-to-eye contact to the other remote viewers, as the video cameras are usually positioned away from the screen.
Consider claim 18. Jiang teaches a system comprising: a memory; and at least one processor (col. 7, lines 8-15), coupled to the memory (claim 10: software and hardware), and operative to: during a videoconference, with a first camera (140, fig. 1a-1c), capturing an image of a first participant reflected from a first viewing screen collocated with the first participant and the first camera, while the first viewing screen is intermittently blacked out (e.g., dark screen period 506, col. 5, line 60 to col. 6, line 28, fig. 5a-c) by causing the first viewing screen to display in a full screen mode (e.g., the entire display screen was darkened out); and providing a sequence of the captured images over a network to at least a second viewing screen of a second participant (col. 5, line 60 to col. 6, line 28, fig. 5a-c).
Jiang does not mention the first viewing screen is intermittently blacked out by causing the first viewing screen to display in a full screen mode to prevent emission of extraneous light during the intermittent black out. However, it would have been obvious to one skilled in the art that darkening a display screen would obviously prevent emission of extraneous light.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Jiang to solve the lacking of eye contacts problem during video conversation, because each of the communicators commonly looks at the video screen and not at the video camera, preventing the communicator from having direct eye-to-eye contact to the other remote viewers, as the video cameras are usually positioned away from the screen.
Consider claim 29. Jiang further teaches the first viewing screen (120, fig 1a-c), coupled to the at least one processor (see claim 10 of Jiang), wherein the first viewing screen does not have a non-reflective coating (e.g., conventional glossy surface, col. 3, lines 44-54).
Consider claim 30. Jiang further teaches the at least one processor includes at least a processor of a desktop computer (e.g., either the laptop or desktop shown in fig. 1c and 2), further comprising: the first camera (140), wherein the first camera is an external web camera and is coupled to the at least one processor (see fig. 1c); and the first viewing screen (120, fig. 1c), coupled to the at least one processor (see claim 10 of Jiang); wherein the desktop computer includes a network interface coupled to the network (the computer 110 in fig 1c inherently comprises a network interface to perform video communication).
Consider claim 31. Jiang further teaches an adjustable mount configured to permit pointing the first camera (see fig. 1C, mount 175, 180).
Claim(s) 3, 20 and 35 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang (9,270,933) in view of Voss (WO 2014/137369).
Consider claims 3 and 20. Jiang does not teach continuously capturing images with the first camera while the first viewing screen is not temporarily blacked out, and discarding those images captured with the first camera while the first viewing screen is not temporarily blacked out.
Voss teaches continuously capturing images with the first camera while the first viewing screen is not temporarily blacked out, and discarding those images captured with the first camera while the first viewing screen is not temporarily blacked out (see fig. 5, steps 510-560; page 11, line 23 through page 12, line 12).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Voss into the teachings of Jiang in order to permit the user to control an image capture device by determining through a user command whether to save a captured video as an image or as a video file. For example, holding the capture button beyond a threshold time indicates the capture should be saved as a video, while releasing the capture button before the threshold time indicates the capture should be saved as an image (page 2, lines 8-14 of Voss).
Consider claim 35. Jiang teaches a method comprising: during a videoconference, with a first camera (140, fig. 1a-1c), capturing an image of a first participant reflected from a first viewing screen collocated with the first participant and the first camera, while the first viewing screen is intermittently blacked out (e.g., dark screen period 506, col. 5, line 60 to col. 6, line 28, fig. 5a-c); and providing a sequence of the captured images [during the blacked out period] over a network to at least a second viewing screen of a second participant (col. 5, line 60 to col. 6, line 28, fig. 5a-c).
Jiang does not teach continuously capturing images with the first camera while the first viewing screen is not temporarily blacked out, and discarding those images captured with the first camera while the first viewing screen is not temporarily blacked out.
Voss teaches continuously capturing images with the first camera while the first viewing screen is not temporarily blacked out, and discarding those images captured with the first camera while the first viewing screen is not temporarily blacked out (see fig. 5, steps 510-560; page 11, line 23 through page 12, line 12).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Voss into the teachings of Jiang in order to permit the user to control an image capture device by determining through a user command whether to save a captured video as an image or as a video file. For example, holding the capture button beyond a threshold time indicates the capture should be saved as a video, while releasing the capture button before the threshold time indicates the capture should be saved as an image (page 2, lines 8-14 of Voss).
Claim(s) 4 and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang (9,270,933) as applied to claim 1 above, and further in view of Kramer et. al (US 20230063441).
Consider claims 4 and 21. Jiang does not expressively teach “further comprising performing keystone correction on the sequence of the periodically captured images.”
However, Kramer does teach further comprising performing keystone correction on the sequence of the periodically captured images (“For example, the improvements may include suppressing noise, normalizing illumination, rejecting motion blur, enhancing resolution, rotating, de-skewing, keystone correction, and/or increasing image size, among others.” Kramer [0073]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Jiang with Kramer to be able to provide the participants in the videoconference with a positive viewing experience. Keystone correction can better align images, ensuring participants are displayed to others in a level, accurate manner.
Claim(s) 5-7, 22-24, 33-34 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang (9,270,933) in view of Oz (WO 2022/238908).
Consider claims 5-6. Jiang does not clearly teach performing color re-saturation comprises applying a machine learning model.
Oz teaches performing the color re-saturation comprises training a machine learning model on pairs of properly saturated and desaturated images; and applying the machine learning model (para 12, para 285).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Oz into the teachings of Jiang in order to correct images that tend to be blurry due to the limited resolution of the camera, motion blur, and video compression.
Consider claim 7. Jiang does not teach training the machine learning model on pairs of properly saturated and desaturated images.
Oz teaches performing the color re-saturation comprises training a machine learning model on pairs of properly saturated and desaturated images; and applying the machine learning model (para 12, para 285).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Oz into the teachings of Jiang in order to correct images that tend to be blurry due to the limited resolution of the camera, motion blur, and video compression.
Consider claims 22-23. Jiang does not clearly teach performing color re-saturation comprises applying a machine learning model.
Oz teaches performing the color re-saturation comprises training a machine learning model on pairs of properly saturated and desaturated images; and applying the machine learning model (para 12, para 285).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Oz into the teachings of Jiang in order to correct images that tend to be blurry due to the limited resolution of the camera, motion blur, and video compression.
Consider claim 24. Jiang does not teach wherein the at least one processor is operative to train the machine learning model on pairs of properly saturated and desaturated images.
Oz teaches performing the color re-saturation comprises training a machine learning model on pairs of properly saturated and desaturated images; and applying the machine learning model (para 12, para 285).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Oz into the teachings of Jiang in order to correct images that tend to be blurry due to the limited resolution of the camera, motion blur, and video compression.
Consider claim 33. Jiang teaches a method comprising: during a videoconference, with a first camera (140, fig. 1a-1c), capturing an image of a first participant reflected from a first viewing screen collocated with the first participant and the first camera, while the first viewing screen is intermittently blacked out (e.g., dark screen period 506, col. 5, line 60 to col. 6, line 28, fig. 5a-c); performing color re-saturation on a sequence of the periodically captured images, gathering the desaturated images with the first camera and the properly saturated images with a second camera directed at the first participant (Herein, the term “dark” is referring to any dark colors which can make a reflective image visible on the display screen. For example, when display screen shows black, deep blue, deep brown, deep green, or deep purple, the reflective image 530 of the local user will show up as illustrated by FIG. 5C. This image can be captured by a video camera and sent to the remote user; col. 5, line 60 to col. 6, line 28, fig. 5a-c); and providing the sequence of the captured images over a network to at least a second viewing screen of a second participant (col. 5, line 60 to col. 6, line 28, fig. 5a-c).
Jiang does not clearly teach performing the color re-saturation comprises training a machine learning model on pairs of properly saturated and desaturated images; and applying the machine learning model.
Oz teaches performing the color re-saturation comprises training a machine learning model on pairs of properly saturated and desaturated images; and applying the machine learning model (para 12, para 285).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Oz into the teachings of Jiang in order to correct images that tend to be blurry due to the limited resolution of the camera, motion blur, and video compression.
Consider claim 34. Jiang teaches a method comprising: during a videoconference, with a first camera (140, fig. 1a-1c), capturing an image of a first participant reflected from a first viewing screen collocated with the first participant and the first camera, while the first viewing screen is intermittently blacked out (e.g., dark screen period 506, col. 5, line 60 to col. 6, line 28, fig. 5a-c); performing color re-saturation on a sequence of the periodically captured images, gathering the desaturated images with the first camera directed at the first viewing screen and gathering the properly saturated images with the first camera directed at the first participant (Herein, the term “dark” is referring to any dark colors which can make a reflective image visible on the display screen. For example, when display screen shows black, deep blue, deep brown, deep green, or deep purple, the reflective image 530 of the local user will show up as illustrated by FIG. 5C. This image can be captured by a video camera and sent to the remote user; col. 5, line 60 to col. 6, line 28, fig. 5a-c); and providing the sequence of the captured images over a network to at least a second viewing screen of a second participant (col. 5, line 60 to col. 6, line 28, fig. 5a-c).
Jiang does not clearly teach performing the color re-saturation comprises training a machine learning model on pairs of properly saturated and desaturated images; and applying the machine learning model.
Oz teaches performing the color re-saturation comprises training a machine learning model on pairs of properly saturated and desaturated images; and applying the machine learning model (para 12, para 285).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Oz into the teachings of Jiang in order to correct images that tend to be blurry due to the limited resolution of the camera, motion blur, and video compression.
Claim(s) 8-9 and 25-26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang (9,270,933) in view of Oz (WO 2022/238908) as applied to claims 1, 5-7, 18, 22-24 above, and further in view of Kraus (12,272,003).
Consider claims 8 and 25. Jiang teaches gathering the desaturated images with the first camera (e.g., dark screen period 506, col. 5, line 60 to col. 6, line 28, fig. 5a-c).
Jiang in view of Oz does not teach gathering the properly saturated images with a second camera directed at the first participant.
Kraus teaches gathering the properly saturated images with the first camera directed at the first participant (see fig. 1, col. 11, lines 1-19).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Kraus into the teachings of Jiang in view of Oz in order to provide modified video image data in such a way that the first user is looking at the corresponding region of the depicted face of the second user.
Consider claims 9 and 26. Jiang teaches gathering the desaturated images with the first camera directed at the first viewing screen (e.g., dark screen period 506, col. 5, line 60 to col. 6, line 28, fig. 5a-c).
Jiang in view of Oz does not teach to gather the properly saturated images with the first camera directed at the first participant.
Kraus teaches gathering the properly saturated images with the first camera directed at the first participant (see fig. 1, col. 11, lines 1-19).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings of Kraus into the teachings of Jiang in view of Oz in order to provide modified video image data in such a way that the first user is looking at the corresponding region of the depicted face of the second user.
Claim(s) 10 and 27 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang as applied to claim 1 above, and further in view of Omori et. al (US 5818399).
Consider claims 10 and 27. Jiang does not expressively teach “further comprising performing image inversion on the sequence of the periodically captured images.”
However, Omori does teach further comprising performing image inversion on the sequence of the periodically captured images (“Therefore, an image processor (not shown) for R screen image performs the inversion of the screen images (R) to output it to the display 112a.”; para 62 of Omori).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Jiang with Omori to again, improve the overall viewing experience of participants. Image orientation is crucial to videoconferencing. In this disclosure wherein pictures are taken from the reflection of display screen, it would be problematic and ensue a poor viewing experience for participants if the transmitted images were oriented in the wrong way.
Claim(s) 16 and 32 is/are rejected under 35 U.S.C. 103 as being unpatentable over Jiang (9,270,933) in view of Baldino et al (US8436888).
Consider claims 16 and 32. Jiang does not teach providing of the sequence of the periodically captured images over the network to at least the second viewing screen of the second participant includes providing the sequence of the periodically captured images over the network to at least a third viewing screen of a third participant, further comprising: interfacing with video conferencing software used for the video conference to determine which one of the first and second participants is speaking; and adjusting the first camera to point to a location on the first viewing screen corresponding to the one of the first and second participants who is speaking.
However, Baldino does teach interfacing with video conferencing software used for the video conference to determine which one of the first and second participants is speaking (“Media switch 20 receives the audio and video packets from participant systems 30, and switch processor 22 analyzes the loudness metric values within the audio packets to determine an active conference speaker.”; para 19 of Baldino); and adjusting the first camera to point to a location on the first viewing screen corresponding to the one of the first and second participants who is speaking (“Once media switch 20 determines the presence of an active speaker, the audio and video information of that active speaker are distributed to the remaining conference participants. In particular, media switch 20 produces and transmits audio and video packets including the audio and video information of the active speaker to the remaining participant systems 30.”; para 19 of Baldino. Once the speaker is detected, the speaker’s video information will be distributed to others, allowing the camera to locate the speaker on the first viewing screen).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Jiang with Baldino to simulate a more realistic videoconferencing environment. In an in-person meeting, it would be natural to focus your attention on the person who is speaking at a given time. Consequently, prioritizing a speaker’s video when he or she is speaking is natural and would make for a more engaging, realistic videoconferencing environment.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUC NGUYEN whose telephone number is (571)272-7503. The examiner can normally be reached 6:30AM-3:45PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at 571-272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DUC NGUYEN
Supervisory Patent Examiner
Art Unit 2691
/DUC NGUYEN/Supervisory Patent Examiner, Art Unit 2691
Prosecution Insights