DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on February 27, 2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is considered by examiner.
Examiner note regarding Drawings
It is the examiner’s opinion any form of photograph image shown in Figures 7, 10 are necessary and “are the only practicable medium for illustrating the claimed invention” per 37 CFR 1.84(b)(1) because the invention pertains to “a video conferencing apparatus that may detect image quality in advance and perform video analysis based on the detected image quality” (specification ¶ [0002]). In each figure, an original image is processed to change details of the image, which may not be captured with sufficient detail in a line drawing to demonstrate the applicant’s invention. Therefore, no drawing objection is raised.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites a method for evaluating a quality of an image detected by a camera of a video conferencing device (generic computer devices to gather and analyze data associated with abstract ideas using mathematical concepts; see MPEP § 2106.04(a)), comprising:
sampling a current frame of an input video stream (considered insignificant pre-solution data gathering activity; see MPEP § 2106.05(g));
extracting image quality information from the current frame (mathematical concept to determine a set of features; see MPEP § 2106.04(a)(2)(I));
comparing the extracted image quality information with reference image quality information generated by an image quality model (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I));
selecting, based on the comparing, an image quality mode of the current frame (considered insignificant post-solution activity based on a mathematical concept; see MPEP § 2106.05(g)); and
proceeding with performing image analysis on the current frame, based on the image quality mode, the image analysis comprising at least one of face recognition and object recognition (considered insignificant well known post-solution activity that is routine and well known activity in the art; see MPEP § 2106.05(g)).
Claim 2 recites the method of claim 1 (as discussed above), wherein the extracting of the image quality information comprises: applying a Fast Fourier Transform (FFT) to the current frame (mathematical concept to determine a set of features; see MPEP § 2106.04(a)(2)(I)); and extracting a high-frequency component from results of the FFT, the high-frequency component having frequencies that are higher than or equal to a reference frequency (mathematical concept to determine a set of features; see MPEP § 2106.04(a)(2)(I)).
Claim 3 recites the method of claim 2 (as discussed above), wherein the extracting of the image quality information further comprises: scaling the high-frequency component by at least one of an absolute value and a log calculation method (mathematical concept to determine a set of features; see MPEP § 2106.04(a)(2)(I)).
Claim 4 recites the method of claim 1 (as discussed above), further comprising: obtaining the reference image quality information from at least one previous frame sampled at a first time point in the input video stream that occurred before a second time point of the current frame in the input video stream (mathematical concept to determine a set of features; see MPEP § 2106.04(a)(2)(I)).
Claim 5 recites the method of claim 4 (as discussed above), further comprising: generating, using the image quality model, the reference image quality information based on previous image quality information extracted from the at least one previous frame (mathematical concept to determine a set of features; see MPEP § 2106.04(a)(2)(I)).
Claim 6 recites the method of claim 5 (as discussed above), further comprising: updating the image quality model based on the image quality mode (considered insignificant well known post-solution activity; see MPEP § 2106.05(g)).
Claim 7 recites the method of claim 1 (as discussed above), further comprising: storing, in an image quality database, the extracted image quality information about the current frame (considered insignificant well known post-solution activity; see MPEP § 2106.05(g)).
Claim 8 recites the method of claim 5 (as discussed above), wherein the selecting of the image quality mode of the current frame comprises: selecting a deferment of judgment mode as the image quality mode of the current frame, based on the extracted image quality information being lower than the reference image quality information and the previous image quality information of the at least one previous frame being higher than the reference image quality information (mathematical concept to select an outcome; see MPEP § 2106.04(a)(2)(I)).
Claim 9 recites a video conferencing device for determining a security mode by processing a video stream provided by a camera (generic computer devices to gather and analyze associated with abstract ideas using mathematical concepts; see MPEP § 2106.04(a)), comprising: a memory storing instructions (generic computer device; see MPEP § 2106.04(a)); and one or more processors communicatively coupled to the memory, wherein the one or more processors are configured to execute the instructions (generic computer devices to gather and analyze associated with abstract ideas using mathematical concepts; see MPEP § 2106.04(a)) to:
sampling a current video frame from the video stream (considered insignificant pre-solution data gathering activity; see MPEP § 2106.05(g));
calculate, using an image quality model, an image quality of the current video frame, the image quality model having been trained with previous video frames of the video stream (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I));
select, based on the image quality of the current video frame, the security mode of the current video frame (considered insignificant post-solution activity based on a mathematical concept; see MPEP § 2106.05(g)); and
perform video analysis of the current video frame based on the security mode and the image quality (considered insignificant well known post-solution activity that is routine and well known activity in the art; see MPEP § 2106.05(g)).
Claim 10 recites the video conferencing device of claim 9 (as discussed above), wherein the one or more processors are further configured to execute the instructions to: calculate a first image quality of the current video frame (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I)); generate the image quality model using previous image quality information of the previous video frames; compare the first image quality with a second image quality generated by the image quality model (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I)); select a state of the camera based on a comparison result between the first image quality and the second image quality (mathematical concept to select an outcome; see MPEP § 2106.04(a)(2)(I)); and update the image quality model with the first image quality based on the state of the camera (considered insignificant post-solution activity based on a mathematical concept; see MPEP § 2106.05(g)).
Claim 11 recites the video conferencing device of claim 10 (as discussed above), wherein the one or more processors are further configured to execute the instructions to: extract contour information from the current video frame (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I)); generate, based on the contour information, the first image quality (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I)); store, in an image quality database, the first image quality (considered insignificant post-solution activity based on a mathematical concept; see MPEP § 2106.05(g)); and generate the image quality model using the previous image quality information of the previous video frames stored in the image quality database (considered insignificant post-solution activity based on a mathematical concept; see MPEP § 2106.05(g)).
Claim 12 recites the video conferencing device of claim 11 (as discussed above), wherein the one or more processors are further configured to execute the instructions to: apply a Fast Fourier Transform (FFT) to the current video frame (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I)); and extract a high-frequency component from results of the FFT, the high-frequency component having frequencies that are higher than or equal to a reference frequency (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I)).
Claim 13 recites the video conferencing device of claim 12 (as discussed above), wherein the one or more processors are further configured to execute the instructions to: scale the high-frequency component by at least one of an absolute value and a log scaling operation (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I)).
Claim 14 recites the video conferencing device of claim 13 (as discussed above), wherein the one or more processors are further configured to execute the instructions to: select the state of the camera based on the comparison result between the first image quality and the second image quality (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I)); and transfer the current video frame to an image analyzer based on the state of the camera (considered insignificant well known post-solution activity after performing a mathematical operation that is routine and well known activity in the art; see MPEP § 2106.05(g)).
Claim 15 recites the video conferencing device of claim 14 (as discussed above), wherein the one or more processors are further configured to execute the instructions to: block transmission of the current video frame to the image analyzer based on the comparison result of the first image quality and the second image quality indicating that the state of the camera is abnormal (considered insignificant well known post-solution activity after performing a mathematical operation that is routine and well known activity in the art; see MPEP § 2106.05(g)).
Claim 16 recites the video conferencing device of claim 10 (as discussed above), wherein the one or more processors are further configured to execute the instructions to: update the image quality model based on a number of the previous video frames constituting the image quality model is less than a reference value (considered insignificant well known post-solution activity after performing a mathematical operation that is routine and well known activity in the art; see MPEP § 2106.05(g)).
Claim 17 recites a method of evaluating a quality of an image transmitted from a camera (generic computer output used to perform functions associated with abstract ideas using mathematical concepts; see MPEP § 2106.04(a)), comprising:
training an image quality model using first image quality information extracted from a plurality of previous video frames sampled from a video stream (mathematical concept to train a model (considered a generic computer component as claimed); see MPEP § 2106.04(a)(2)(I) and considered routine and well known activity in the art; see MPEP § 2106.05(g));
generating, using the image quality model, reference image quality information (mathematical concept to determine a set of features; see MPEP § 2106.04(a)(2)(I));
extracting second image quality information from a current video frame sampled from the video stream, the current video frame corresponding to a time point that occurs after previous time points corresponding to the plurality of previous video frames (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I); and
selecting a quality mode of the current video frame by comparing the second image quality information with the reference image quality information (considered insignificant post-solution activity based on a mathematical concept; see MPEP § 2106.05(g).
Claim 18 recites the method of claim 17 (as discussed above), wherein the quality mode comprises at least one of: a first mode indicating that training of the image quality model is completed and that the second image quality information exceeds the reference image quality information; a second mode indicating that training of the image quality model is completed, that the second image quality information fails to meet the reference image quality information, and that the first mode is assigned to at least one previous video frame of the plurality of previous video frames; a third mode indicating that training of the image quality model is completed, that the second image quality information fails to meet the reference image quality information, and that the first mode is not assigned to the plurality of previous video frames; and a fourth mode indicating that training of the image quality model is incomplete (each mode represents a given result from a mathematical output, thereby interpreted as post-solution activity after performing a mathematical operation; see MPEP § 2106.05(g)).
Claim 19 recites the method of claim 17 (as discussed above), wherein the extracting of the second image quality information comprises: generating the second image quality information using at least one of a high-frequency region modeling technique, an edge modeling technique, and motion estimation operations (mathematical concept to determine a subset of features; see MPEP § 2106.04(a)(2)(I)).
Claim 20 recites the method of claim 17 (as discussed above), further comprising: updating the image quality model with the first image quality information extracted from the current video frame based on the quality mode (considered insignificant well known post-solution activity after performing a mathematical operation that is routine and well known activity in the art; see MPEP § 2106.05(g)).
The claimed invention is directed to an abstract idea without significantly more. The claims recite mathematical concepts as outlined above and described in the MPEP 2106.04(a)(2)(I) with select limitations directed to extra-solution activity under MPEP 2106.04(g). MPEP 2106.04(a)(2)(I) states:
It is important to note that a mathematical concept need not be expressed in mathematical symbols, because "[w]ords used in a claim operating on data to solve a problem can serve the same purpose as a formula." In re Grams, 888 F.2d 835, 837 and n.1, 12 USPQ2d 1824, 1826 and n.1 (Fed. Cir. 1989). See, e.g., SAP America, Inc. v. InvestPic, LLC, 898 F.3d 1161, 1163, 127 USPQ2d 1597, 1599 (Fed. Cir. 2018) (holding that claims to a ‘‘series of mathematical calculations based on selected information’’ are directed to abstract ideas); Digitech Image Techs., LLC v. Elecs. for Imaging, Inc., 758 F.3d 1344, 1350, 111 USPQ2d 1717, 1721 (Fed. Cir. 2014) (holding that claims to a ‘‘process of organizing information through mathematical correlations’’ are directed to an abstract idea); and Bancorp Servs., LLC v. Sun Life Assurance Co. of Can. (U.S.), 687 F.3d 1266, 1280, 103 USPQ2d 1425, 1434 (Fed. Cir. 2012) (identifying the concept of ‘‘managing a stable value protected life insurance policy by performing calculations and manipulating the results’’ as an abstract idea).
Therefore, if a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation based on mathematical concepts but for the recitation of generic computer components, then it falls within the “Mathematical Concepts” grouping of abstract ideas. Accordingly, these claims each recite an abstract idea.
This judicial exception is not integrated into a practical application. The computer components are recited at a high-level of generality (i.e., generic computer components (memory, processor, and instructions including generic models for performing a general function of calculating quality metrics from image data, which is described with a high level of generality of automating a mathematical operation) such that it amounts to no more than mere instructions to apply the exception using a generic computer component. Accordingly, the computer components do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Therefore, the aforementioned claims are directed to abstract ideas.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a generic placeholder-related computer components, the memory to store instructions executed on a processor to perform mathematical calculations to determine mathematical relationships amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an invention concept. The claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-7, 17, 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al (US 2022/0036535) in view of New et al (US 2011/0026593).
Regarding Claim 1, Lee et al teach a method for evaluating a quality of an image detected by a camera of a video conferencing device (image quality assessment method of video stream data from video stream apparatus 100; Fig 1, 2 and ¶ [0023]), comprising:
sampling a current frame of an input video stream (blocks for an image in a video stream are formed, step S210, where the image samples are based on blocks of information; Fig 2 and ¶ [0023]-[0024]);
extracting image quality information from the current frame (the image blocks are input to a quality assessment model to assess the quality of the image block, S220, and the quality assessment model extracts image features from the block of the image frame of the video stream, step 230; Fig 2 and ¶ [0025]-[0026], [0036]);
comparing the extracted image quality information with reference image quality information generated by an image quality model (the extracted block image features are identified based on an objective function to determine the index of the image quality using the quality assessment model, step 230 and Fig 2, 4 and ¶ [0036], [0039]); and
proceeding with performing image analysis on the current frame, based on the image quality mode, the image analysis comprising at least one of face recognition and object recognition (image feature analysis of the blocks can correspond to a type of facial feature, such as a human face, eyes or mouth; Fig 2, 4 and ¶ [0036]).
Lee et al does not teach selecting, based on the comparing, an image quality mode of the current frame.
New et al is analogous art pertinent to the technological problem addressed in the current application and teaches selecting, based on the comparing, an image quality mode of the current frame (the selecting unit 14 of the image processing apparatus 100a switches (SW1, SW2) to either a first processing mode (low resolution decoding mode) or second processing mode (full resolution decoding mode) based on a mode identifier, determined based on the number of reference frames, and selects the processing for decoding the coded (input) image; Fig 12 and ¶ [0204]-[0209]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al with New et al including selecting, based on the comparing, an image quality mode of the current frame. By implementing a selection for an image processing mode to process image data at either low resolution or high resolution, the memory bandwidth and capacity required for the given frame is reduced while concurrently preventing image quality degradation, as recognized by New et al (¶ [0034]-[0035]).
Regarding Claim 4, Lee et al in view of New et al teach the method of claim 1 (as described above), further comprising: obtaining the reference image quality information from at least one previous frame sampled at a first time point in the input video stream that occurred before a second time point of the current frame in the input video stream (New et al, a plurality of sequenced coded images are included in the reference frames (sequence parameter set), stored in the frame memory 108, and are considered when decoding a coded image within the sequence (thereby the reference image quality is sampled from a time point prior to the time point analyzed of the frame being decoded); Fig 12 and ¶ [0205]).
Regarding Claim 5, Lee et al in view of New et al teach the method of claim 4 (as described above), further comprising: generating, using the image quality model (“image quality model” described broadly as a means to detect an edge or motion within the image, specification ¶ [0079]-[0080]), the reference image quality information based on previous image quality information extracted from the at least one previous frame (New et al, the reference frames include the resolution (quality information for motion analysis) and is determined for inter-prediction coding (thereby the reference image quality is sampled from a time point prior to the time point analyzed of the frame being decoded); Fig 12 and ¶ [0205]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al with New et al including generating, using the image quality model, the reference image quality information based on previous image quality information extracted from the at least one previous frame. By implementing an up-sampling unit based on an image processing mode, an image may be processed at either low resolution or high resolution, and implement a means to replace deleted frequency information to down-sampled images and avoid distortions from unnecessary upsampling, as recognized by New et al (¶ [0038]-[0039]).
Regarding Claim 6, Lee et al in view of New et al teach the method of claim 5 (as described above), further comprising: updating the image quality model based on the image quality mode (New et al, the up-sampling unit 109 receives the decoding mode based on the selecting unit 14 and the given mode is updated for processing the image date to generate the decoded image; Fig 12 and ¶ [0207]-[0209]).
Regarding Claim 7, Lee et al in view of New et al teach the method of claim 1 (as described above), further comprising: storing, in an image quality database, the extracted image quality information about the current frame (Lee et al, the processor 170 can store a mean opinion score or differential mean opinion score (image quality data) in an image database, used for training the quality assessment model; ¶ [0028]-[0029]).
Regarding Claim 17, Lee et al teach a method of evaluating a quality of an image transmitted from a camera (image quality assessment method of video stream data using a video stream apparatus 100, with video generated from apparatus 50, such as a camera; Fig 1, 2 and ¶ [0016], [0023]), comprising:
training an image quality model using first image quality information extracted from a plurality of previous video frames sampled from a video stream (a training method of an image quality assessment model is performed with streaming images selected and divided into blocks, S310, which are used as learning samples for training the model; Fig 3 and ¶ [0027]-[0028]);
generating, using the image quality model, reference image quality information (the model generates perception assessment scores of the models and rating results of human perception to generate quality assessment by the model, S330, as well as an objective function, S350; Fig 3 and ¶ [0029]-[0034]); and
extracting second image quality information from a current video frame sampled from the video stream, the current video frame corresponding to a time point that occurs after previous time points corresponding to the plurality of previous video frames (interpreted that the “current video frame” is analyzed with a trained model and not performing additional training; (the image blocks are input to a quality assessment model to assess the quality of the image block, S220, and the quality assessment model extracts image features from the block of the image frame of the video stream, step 230; Fig 2 and ¶ [0025]-[0026], [0036]).
Lee et al does not teach selecting a quality mode of the current video frame by comparing the second image quality information with the reference image quality information.
New et al is analogous art pertinent to the technological problem addressed in the current application and teaches selecting a quality mode of the current video frame by comparing the second image quality information with the reference image quality information (the selecting unit 14 of the image processing apparatus 100a switches (SW1, SW2) to either a first processing mode (low resolution decoding mode) or second processing mode (full resolution decoding mode) based on a mode identifier, determined based on the number of reference frames, and selects the processing for decoding the coded (input) image; Fig 12 and ¶ [0204]-[0209]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al with New et al including selecting a quality mode of the current video frame by comparing the second image quality information with the reference image quality information. By implementing a selection for an image processing mode to process image data at either low resolution or high resolution, the memory bandwidth and capacity required for the given frame is reduced while concurrently preventing image quality degradation, as recognized by New et al (¶ [0034]-[0035]).
Regarding Claim 19, Lee et al in view of New et al teach the method of claim 17 (as described above), wherein the extracting of the second image quality information comprises: generating the second image quality information using at least one of a high-frequency region modeling technique, an edge modeling technique, and motion estimation operations (New et al, the quality information can include data for motion analysis, used by the motion compensation unit 110; Fig 12 and ¶ [0211]-[0214]).
Regarding Claim 20, Lee et al in view of New et al teach the method of claim 17 (as described above), further comprising: updating the image quality model (“image quality model” described broadly as a means to detect an edge or motion within the image, specification ¶ [0079]-[0080]) with the first image quality information extracted from the current video frame based on the quality mode (New et al, the up-sampling unit 109 receives the decoding mode based on the selecting unit 14 and the given mode is updated for processing the image date to generate the decoded image; Fig 12 and ¶ [0207]-[0209]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al with New et al including updating the image quality model with the first image quality information extracted from the current video frame based on the quality mode. By updating the reference data used for decoding image data, a means to replace deleted frequency information to down-sampled images and avoid distortions from unnecessary upsampling is achieved, thereby reducing memory while improving the quality in the output decoded image data, as recognized by New et al (¶ [0036]-[0039]).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Lee et al (US 2022/0036535) in view of New et al (US 2011/0026593) and Yamamoto et al (US 2011/0002552).
Regarding Claim 2, Lee et al in view of New et al teach the method of claim 1 (as described above).
Lee et al in view of New et al does not teach wherein the extracting of the image quality information comprises: applying a Fast Fourier Transform (FFT) to the current frame; and extracting a high-frequency component from results of the FFT, the high-frequency component having frequencies that are higher than or equal to a reference frequency.
Yamamoto et al is analogous art pertinent to the technological problem addressed in the current application and teaches wherein the extracting of the image quality information comprises: applying a Fast Fourier Transform (FFT) to the current frame (a FFT is performed to the image frame by the frequency characteristics analyzer 23; Fig 3 and ¶ [0092]); and extracting a high-frequency component from results of the FFT (a non-linear filtering processor 25 is performed on the frame data to remove noise and obtain edges and boundaries of objects (high-frequency components); Fig 3 and ¶[0094]-[0096]), the high-frequency component having frequencies that are higher than or equal to a reference frequency (a cutoff frequency process is used to determine the frequency limit applied in the filtering process; ¶ [0099]-[0100]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al in view of New et al with Yamamoto et al including wherein the extracting of the image quality information comprises: applying a Fast Fourier Transform (FFT) to the current frame; and extracting a high-frequency component from results of the FFT, the high-frequency component having frequencies that are higher than or equal to a reference frequency. By using a FFT, signal noise is easily identified and may be quickly removed or minimized, thereby improving the quality of the processed image, as recognized by Yamamoto et al (¶ [0024]).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Lee et al (US 2022/0036535) in view of New et al (US 2011/0026593), Yamamoto et al (US 2011/0002552) and Kogure et al (US 2006/0204030).
Regarding Claim 3, Lee et al in view of New et al and Yamamoto et al teach the method of claim 2 (as described above).
Lee et al in view of New et al and Yamamoto et al do not teach scaling the high-frequency component by at least one of an absolute value and a log calculation method.
Kogure et al is analogous art pertinent to the technological problem addressed in the current application and teaches scaling the high-frequency component by at least one of an absolute value and a log calculation method (the FFT data undergoes a transform based on a logarithmic calculation; ¶ [0119]-[0121]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al in view of New et al and Yamamoto et al with Kogure et al including scaling the high-frequency component by at least one of an absolute value and a log calculation method. By performing an amplitude (scaling) step, a higher-quality of the digital signal can be detected, as recognized by Kogure et al (¶ [0027]-[0028]).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Lee et al (US 2022/0036535) in view of Du et al (CN 115720257).
Regarding Claim 9, Lee et al teach a video conferencing device for determining a security mode by processing a video stream provided by a camera (video streaming system 1 with image quality assessment apparatus 100 from video stream data generated by apparatus 50, such as a camera; Fig 1 and ¶ [0015]-[0016]), comprising:
a memory storing instructions (image quality assessment apparatus 100 includes a memory 120 that stores instructions; Fig 1 and ¶ [0017], [0019]); and
one or more processors communicatively coupled to the memory, wherein the one or more processors are configured to execute the instructions (image quality assessment apparatus 100 includes a processor 170, coupled to memory 120, that executes instructions; Fig 1 and ¶ [0017], [0021]) to:
sample a current video frame from the video stream (blocks for an image in a video stream are formed, step S210, where the image samples are based on blocks of information; Fig 2 and ¶ [0023]-[0024]);
calculate, using an image quality model, an image quality of the current video frame (the image blocks are input to a quality assessment model to assess the quality of the image block, S220, and the quality assessment model extracts image features from the block of the image frame of the video stream, step 230; Fig 2, 4 and ¶ [0025]-[0026], [0036], [0039]), the image quality model having been trained with previous video frames of the video stream (the image quality assessment model is trained with streaming images selected and divided into blocks, S310, which are used as learning samples for training the model; Fig 3 and ¶ [0027]-[0028]); and
perform video analysis of the current video frame based on the security mode and the image quality (image feature analysis of the blocks can correspond to a type of facial feature, such as a human face, eyes or mouth; Fig 2, 4 and ¶ [0036]).
Lee et al does not teach determining a security mode; and to select, based on the image quality of the current video frame, the security mode of the current video frame.
Du et al is analogous art pertinent to the technological problem addressed in the current application and teaches determining a security mode (transmission mode based on security needs; ¶ [0020]-[0021], [0026]-[0027], [0048]); and to select, based on the image quality of the current video frame, the security mode of the current video frame (the quality of video frame (video code rate, resolution and frame rate) is determined based on the in the channel and used for selecting the transmission mode as applied to a security management system; Fig 1, 3, 4 and ¶ [0020]-[0021], [0025]-[0027], [0048]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al with Du et al including determining a security mode; and to select, based on the image quality of the current video frame, the security mode of the current video frame. By determining a security mode, control of the video bit rate transmission may be determined, resulting in improved quality of experience for users and technical adaptability of the video parameters during real-time video transmission, as recognized by Du et al (¶ [0003]-[0005]).
Claims 10-11, 16 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al (US 2022/0036535) in view of Du et al (CN 115720257) and Finn et al (US 2009/0040303, disclosed in IDS 02/27/2024).
Regarding Claim 10, Lee et al in view of Du et al teach the video conferencing device of claim 9 (as described above), including the one or more processors are further configured to execute the instructions (Lee et al, image quality assessment apparatus 100 includes a processor 170, coupled to memory 120, that executes instructions; Fig 1 and ¶ [0017], [0021]) to calculate a first image quality of the current video frame (Lee et al, the image blocks are input to a quality assessment model to assess the quality of the image block, S220, and the quality assessment model extracts image features from the block of the image frame of the video stream, step 230; Fig 2 and ¶ [0025]-[0026], [0036]);
generate the image quality model using previous image quality information of the previous video frames (Lee et al, the image quality assessment model is generated S370, based on learning data, S310 analyzed using a perception assessment score S330 and objective function S350; Fig 3 and ¶ [0027]-[0035]); and
compare the first image quality with a second image quality generated by the image quality model (Lee et al, the image blocks are input to a quality assessment model to assess the quality of the image block, S220, and the quality assessment model extracts image features from the block of the image frame of the video stream, step 230; Fig 2 and ¶ [0025]-[0026], [0036]);
Lee et al in view of Du et al do not explicitly teach to select a state of the camera based on a comparison result between the first image quality and the second image quality; and update the image quality model with the first image quality based on the state of the camera.
Finn et al is analogous art pertinent to the technological problem addressed in the current application and teaches to select a state of the camera based on a comparison result between the first image quality and the second image quality (image quality metrics may be calculated from a single camera or may be calculated from a number of cameras focused on a similar ROI; ¶ [0019]-[0020]); and update the image quality model with the first image quality based on the state of the camera (the video quality metric alarm thresholds of the video quality detection system (VQD) 36 may be updated when the fused video quality metrics indicate camera issues (out of focus or obscuration) but determined to not warrant repair; ¶ [0018]-[0019]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al in view of Du et al with Finn et al including to select a state of the camera based on a comparison result between the first image quality and the second image quality; and update the image quality model with the first image quality based on the state of the camera. By performing different metric analysis of image data, a more accurate assessment of a potential camera issue, such as focus, is easily detected and remedies, as recognized by Finn et al (¶ [0019]-[0020]).
Regarding Claim 11, Lee et al in view of Du et al and Finn et al teach the video conferencing device of claim 10 (as described above), wherein the one or more processors are further configured to execute the instructions to: extract contour information from the current video frame (Lee et al, the quality information can include edge pixel analysis (vectors with orientation for feature detection ¶ [0042]) and used to identify features (identify blocks corresponding to types of facial features, thereby contours identified in the features); Fig 2 and ¶ [0026], [0036]-[0037]); generate, based on the contour information, the first image quality (Lee et al, the features are analyzed for an image quality assessment, resulting in a mean opinion score or differential mean opinion score (image quality data); ¶ [0026]-[0027]); store, in an image quality database, the first image quality (Lee et al, the processor 170 can store the mean opinion score or differential mean opinion score (image quality data) in an image database; ¶ [0028]-[0029]); and generate the image quality model using the previous image quality information of the previous video frames stored in the image quality database (Lee et al, the processor 170 can use the mean opinion score or differential mean opinion score (image quality data) for training the quality assessment model; ¶ [0028]-[0029]).
Regarding Claim 16, Lee et al in view of Du et al and Finn et al teach the video conferencing device of claim 10 (as described above), wherein the one or more processors are further configured to execute the instructions to: update the image quality model based on a number of the previous video frames constituting the image quality model is less than a reference value (Finn et al, the video quality metric alarm thresholds of the video quality detection system (VQD) 36 may be updated (updating the image quality model) when the fused video quality metrics indicate camera issues (quality value less than reference value) but determined to not warrant repair; ¶ [0018]-[0019]).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Lee et al (US 2022/0036535) in view of Du et al (CN 115720257), Finn et al (US 2009/0040303, disclosed in IDS 02/27/2024) and Yamamoto et al (US 2011/0002552).
Regarding Claim 12, Lee et al in view of Du et al and Finn et al teach the video conferencing device of claim 11 (as described above), including the one or more processors configured to execute the instructions (Lee et al, image quality assessment apparatus 100 includes a processor 170, coupled to memory 120, that executes instructions; Fig 1 and ¶ [0017], [0021]).
Lee et al in view of Du et al and Finn et al do not teach to apply a Fast Fourier Transform (FFT) to the current video frame; and extract a high-frequency component from results of the FFT, the high-frequency component having frequencies that are higher than or equal to a reference frequency.
Yamamoto et al is analogous art pertinent to the technological problem addressed in the current application and teaches to apply a Fast Fourier Transform (FFT) to the current video frame (a FFT is performed to the image frame by the frequency characteristics analyzer 23; Fig 3 and ¶ [0092]); and extract a high-frequency component from results of the FFT (a non-linear filtering processor 25 is performed on the frame data to remove noise and obtain edges and boundaries of objects (high-frequency components); Fig 3 and ¶[0094]-[0096]), the high-frequency component having frequencies that are higher than or equal to a reference frequency (a cutoff frequency process is used to determine the frequency limit applied in the filtering process; ¶ [0099]-[0100]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al in view of Du et al and Finn et al with Yamamoto et al including to apply a Fast Fourier Transform (FFT) to the current video frame; and extract a high-frequency component from results of the FFT, the high-frequency component having frequencies that are higher than or equal to a reference frequency. By using a FFT, signal noise is easily identified and may be quickly removed or minimized, thereby improving the quality of the processed image, as recognized by Yamamoto et al (¶ [0024]).
Claims 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al (US 2022/0036535) in view of Du et al (CN 115720257), Finn et al (US 2009/0040303, disclosed in IDS 02/27/2024), Yamamoto et al (US 2011/0002552) and Kogure et al (US 2006/0204030).
Regarding Claim 13, Lee et al in view of Du et al and Finn et al with Yamamoto et al teach the video conferencing device of claim 11 (as described above), including the one or more processors configured to execute the instructions (Lee et al, image quality assessment apparatus 100 includes a processor 170, coupled to memory 120, that executes instructions; Fig 1 and ¶ [0017], [0021]).
Lee et al in view of Du et al and Finn et al with Yamamoto et al do not teach to scale the high-frequency component by at least one of an absolute value and a log scaling operation.
Kogure et al is analogous art pertinent to the technological problem addressed in the current application and teaches to scale the high-frequency component by at least one of an absolute value and a log scaling operation (the FFT data undergoes a transform based on a logarithmic calculation; ¶ [0119]-[0121]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al in view of New et al and Yamamoto et al with Kogure et al including to scale the high-frequency component by at least one of an absolute value and a log scaling operation. By performing an amplitude (scaling) step, a higher-quality of the digital signal can be detected, as recognized by Kogure et al (¶ [0027]-[0028]).
Regarding Claim 14, Lee et al in view of Du et al, Finn et al, Yamamoto et al and Kogure et al teach the video conferencing device of claim 13 (as described above), wherein the one or more processors are further configured to execute the instructions to: select the state of the camera based on the comparison result between the first image quality and the second image quality (Finn et al, image quality metrics may be calculated from a single camera may be calculated from a number of cameras focused on a similar ROI; ¶ [0019]-[0020]); and update the image quality model with the first image quality based on the state of the camera (Finn et al, the video quality metric alarm thresholds may be updated (image quality model) when the fused video quality metrics indicate camera issues (out of focus or obscuration) but determined to not warrant repair; ¶ [0018]-[0019]).
It would have been obvious before the effective filing date of the current application to combine the teachings of Lee et al in view of Du et al, Finn et al, Yamamoto et al and Kogure et al including to select a state of the camera based on a comparison result between the first image quality and the second image quality; and update the image quality model with the first image quality based on the state of the camera. By performing different metric analysis of image data, a more accurate assessment of a potential camera issue, such as focus, is easily detected and remedies, as recognized by Finn et al (¶ [0019]-[0020]).
Regarding Claim 15, Lee et al in view of Du et al, Finn et al, Yamamoto et al and Kogure et al teach the video conferencing device of claim 14 (as described above), wherein the one or more processors are further configured to execute the instructions to: block transmission of the current video frame to the image analyzer based on the comparison result of the first image quality and the second image quality indicating that the state of the camera is abnormal (Finn et al, the camera data is analyzed including to determine if the camera is out of focus (abnormal) the video quality metric alarm thresholds may be updated (image quality model) when the fused video quality metrics indicate camera issues (out of focus or obscuration); ¶ [0018]-[0019]).
Allowable Subject Matter
Claims 8, 18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The examiner notes the claims are subject to overcoming the 35 U.S.C. § 101 rejections (discussed above) and the following objections pertain to not being subject to a prior art rejection.
Regarding Claim 8, the following limitations, in combination with the claims in which it depends, are identified as novel over the prior art:
Claim 8. The method of claim 5, wherein the selecting of the image quality mode of the current frame comprises: selecting a deferment of judgment mode as the image quality mode of the current frame, based on the extracted image quality information being lower than the reference image quality information and the previous image quality information of the at least one previous frame being higher than the reference image quality information.
Regarding Claim 18, the following limitations, in combination with the claims in which it depends, are identified as novel over the prior art:
Claim 18. The method of claim 17, wherein the quality mode comprises at least one of: a first mode indicating that training of the image quality model is completed and that the second image quality information exceeds the reference image quality information; a second mode indicating that training of the image quality model is completed, that the second image quality information fails to meet the reference image quality information, and that the first mode is assigned to at least one previous video frame of the plurality of previous video frames; a third mode indicating that training of the image quality model is completed, that the second image quality information fails to meet the reference image quality information, and that the first mode is not assigned to the plurality of previous video frames; and a fourth mode indicating that training of the image quality model is incomplete.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kim et al (US 2024/0370541, application US 18/421,554), from the same applicant and co-inventors, and teach a method of analyzing an image for a security model based on analysis of at least one face in the image, which is distinguishable by focusing on the security mode based on analyzing detection of objects, counts of people, counts of faces and authorization factors associated with the primary face identified in the image.
Park et al (Structural Similarity based Image Compression for LCD Overdrive) teach an encoder mode includes six different compression modes, based on the size of the code bit, including a skip mode applied for the image data block when the image block is less than a bit minimum threshold.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATHLEEN M BROUGHTON whose telephone number is (571)270-7380. The examiner can normally be reached Monday-Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATHLEEN M BROUGHTON/Primary Examiner, Art Unit 2661