Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see pg. 8-10 of applicant’s arguments, filed 02/27/2026, with respect to the rejection(s) of claim(s) 1-5,7-9 and 11-20 under 103 have been fully considered and are persuasive. For the original claim limitations, the original art is used. However, for the amended limitations that are underlines below, the arguments about how the limitations differ from the prior art is persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Herz and Jang.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7-8, 13, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ju (Pub No. US 20130315556 A1) in view of Herz (US 20130169834 A1) and further in view of Zavesky (Pub No. US 20160092751 A1) and further in view of Jang (US 20200046460 A1).
As per claim 1, Ju teaches the claimed:
A non-transitory computer readable medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising: (Ju fig. 1 shows circuits arranged to process features of a digital image for analysis. This necessarily requires a computer readable medium and instructions to executed.).
Ju alone does not explicitly teach the remaining claim limitations.
However, Ju in combination with Herz teaches the claimed:
capturing a video comprising a plurality of frames of a face of an individual;
determining that the video fails to satisfy one or more quality criteria, wherein the one or more quality criteria comprise at least one of: a face angle criterion, a head pose criterion, a head movement speed criterion, a head position criterion, a camera stability criterion, a camera defocus criterion, a mouth shape criterion, or a jaw pose criterion; (Herz [0022]: “Further alternatives to the step 204 selection of a center-of-mass frame include the following. The user may use the GUI 104 to select a single frame based on either the composition of the frame or the timing of the sequence. Alternatively, the user may use the GUI 104 to select a single frame as a first approximation for the center-of-mass frame based on composition or timing (e.g., the image in the frame is compelling in some way as it relates to the image content and/or to a particular moment in time). It may be, however, that the first approximation for the center-of-mass frame has an image quality that is less than desired. For example, the subject matter may not be properly lit, off-centered, clipped and/or blurry. To remedy this, the processing unit 101 may select a nearby frame, as a second center-of-mass frame, which may also have the preferred characteristics of the first center-of-mass frame, but with improved quality (e.g., absence of motion blur and other artifacts). Alternatively, there may be no user intervention in the initial approximation. Instead, the processing unit 101 may select one or more various frames of interest based on a quality parameter, such as where detected eyes of a face in the image are opened, centering of the image subject, size of a detected face, a detected face directly facing the camera or indirectly facing the camera, brightness, and so on. In this alternative, the processing unit 101 is configured as a non-transitory medium having stored instructions, that upon execution, perform algorithms to determine the above quality parameters.” This is an angle criterion because determination of if a face is visible requires face detection. The face angle can be in relation to the camera to determine quality for a particular analytical goal.).
Ju alone does not explicitly teach the remaining claim limitations.
However, Ju in combination with Zavesky teaches the claimed:
and providing guidance of one or more actions to be performed by the individual to cause an updated video to satisfy the one or more quality criteria.
(Zavesky teaches indicating when an image is low-quality, and provides guidance changing the conditions of capturing the image to obtain better quality video. Zavesky [0038]: “Responsive to receiving the image data 110, the image processing system 102 may make a determination of a quality of a representation of one or more objects based on the image data 110. In particular, the image processing system 102 may identify a particular object based on the particular image capture setting and provide a notification as to the quality of the particular object. For example, when the particular image capture setting is the portrait setting, the image processing system 102 may identify a person (e.g., a face) based on the image data 110 and initiate a notification that indicates a quality of the representation of the person within the captured image. To illustrate, the notification may indicate that the image of the person is a high quality image. Alternatively, the notification may indicate that the image of the person is poor quality (e.g., blurry or low-lit). Additionally or alternatively, the notification may suggest an alternate image capture setting that may provide a better quality image of the person. In some embodiments, the image processing system 102 may adjust the image capture settings of the camera and prompt a user of the camera to capture another image using the adjusted settings. For example, the image processing system 102 may be configured to automatically adjust the image capture setting or may request permission from the user to adjust the image capture settings.” The implementation of an alternate image capture setting causes the updated image.).
Ju alone does not explicitly teach the remaining claim limitations.
However, Ju in combination with Jang teaches the claimed:
criteria, wherein the one or more actions comprise at least one of: move head, rotate head, change facial expression, or slow down head movement. (Jang [0015]: “The computing device may receive a stream of 3D images, and may generate modified versions of each image in the stream of 3D images. Accordingly, as a user moves their head, changes their facial expressions, etc., the computing device generates updated modified 3D images that reflect the current facial expressions, head positions, and so on. If the computing device includes an image sensor to generate the stream of 3D images and a display device to display the modified versions of the stream of 3D images, then the computing device can act as a virtual mirror that shows the person their post-treatment face in real time, including their teeth, facial contours (e.g., lip support, changes to smile line, etc.), soft tissues, and so on.” Because the motion of the head changes the output of the model and shows different features that might make a relevant treatment, it would be obvious to deliberately move the head to achieve a more accurate model for the user’s purpose.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use adjustment of the position of a face in the viewpoint of a camera as a quality criterion as taught by Herz with the system of Ju in order to allow the user to seek a specific angle and a specific perspective from which to view an image so that the image depicts a train the user desires to measure.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the suggestions for changes to capture high-quality images as taught by Zavesky with the system of Ju in order to make recommendations about how to improve images instead of just removing the ones that do not meet quality standards.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the repositioning of a head relative to the camera to create a more accurate 3D model as taught by Jang with the system of Ju in view of Herz in order to allow the user to move the head’s position to gain a certain perspective on the object according to a quality criterion the user may have.
As per claim 17, this claim is similar in scope to limitations recited in claim 1, and thus is rejected under the same rationale.
As per claim 2, Ju teaches the claimed:
2. The non-transitory computer readable medium of claim 1, the operations further comprising:
capturing the updated video comprising a second plurality of frames of the face of the individual after providing the guidance;
and determining that the updated video satisfies the one or more quality criteria. (Ju claim 13: “13. A video recording apparatus of recording an output video sequence for an image capture module, comprising: an input circuit, arranged for deriving a first video sequence from an input video sequence generated by the image capture module, wherein the first video sequence is composed of a plurality of video frames of the first video sequence; an image quality estimation circuit, arranged for calculating an image quality metric value for each of the video frames; a selection circuit, arranged for referring to the image quality metric value to select or drop each of the video frames of the first video sequence, and accordingly obtaining a second video sequence composed of selected video frames; and an output circuit, arranged for generating the recorded output video sequence according to the second video sequence.”Ju teaches making adjustments to an image and then outputting the sequence of videos with images that are satisfactory according to the quality standards. This determines that the video meets the criteria. The output video is the second plurality of frames.).
As per claim 18, this claim is similar in scope to limitations recited in claim 2, and thus is rejected under the same rationale.
As per claim 3, Ju teaches the claimed:
3. The non-transitory computer readable medium of claim 2, the operations further comprising:
determining that one or more frames of the second plurality of frames of the updated video fail to satisfy the one or more quality criteria;
and removing the one or more frames from the updated video. (Ju teaches checking video frames for quality and dropping those that don’t meet it. Ju [0040]: “FIG. 4 is a diagram illustrating a third video recording example based on the proposed video recording apparatus 100 shown in FIG. 1. In this example, the input circuit 102 directly outputs the input video sequence V_IN as the first video sequence V_1 composed of video frames F.sub.1-F.sub.10, where the frame rate of the input video sequence V_IN is 120 Hz. As the video frames F.sub.2-F.sub.4 and F.sub.6-F.sub.9 include blurry image contents, the corresponding image quality metric values calculated by the image quality estimation circuit 104 would indicate that the video frames F.sub.2-F.sub.4 and F.sub.6-F.sub.9 have worse quality. Thus, the selection circuit 106 generates the second video sequence V_2 by selecting video frames F.sub.1, F.sub.5, F.sub.10 and dropping video frames F.sub.2-F.sub.4, F.sub.6-F.sub.9. In this example, the target frame rate of the output video sequence V_OUT is 30 Hz, which is lower than the frame rate of the input video sequence V_IN. Though the frame rate of the second video sequence V_2 is equal to the target frame rate of the output video sequence V_OUT due to the proposed image quality based video frame selection scheme, the interval between the image capture timing of the video frames F.sub.5 and F.sub.10 is not equal to an expected interval between image display timing of consecutive video frames (e.g., 1/30 second). …” Ju teaches dropping the frames of poor quality from the original video to make a second sequence.).
As per claim 19, this claim is similar in scope to limitations recited in claim 3, and thus is rejected under the same rationale.
As per claim 4, Ju teaches the claimed:
4. The non-transitory computer readable medium of claim 3, the operations further comprising: generating a replacement frame for at least one removed frame, wherein the replacement frame is generated based on a first frame preceding the removed frame and a second frame following the removed frame and comprises an intermediate state of the face between a first state of the face in the first frame and a second state of the face in the second frame. (Ju teaches interpolating a replacement for a removed frame based on at least one adjacent frame. These can include the preceding and following frames. Ju [0040]: “…and adding a new video frame F.sub.9' to the second video sequence V_2, where the interval between the image capture timing of the video frames F.sub.5 and F.sub.9' is equal to an expected interval between image display timing of consecutive video frames (e.g., 1/30 second). In one exemplary design, the video frame F.sub.9' is interpolated based on at least one of the adjacent selected video frames F.sub.5 and F.sub.10 in the second video sequence V_2. The video frame interpolation may adjust the weighting factors of the referenced video frames to obtain an interpolated video frame with good quality. …”).
As per claim 20, this claim is similar in scope to limitations recited in claim 4, and thus is rejected under the same rationale.
As per claim 7, Ju alone does not explicitly teach the claimed limitations.
However, Ju in combination with Zavesky teaches the claimed:
7. The non-transitory computer readable medium of claim 1, the operations further comprising: outputting a notice of the one or more quality criteria prior to beginning capturing of the video. (Zavesky [0038]: “Responsive to receiving the image data 110, the image processing system 102 may make a determination of a quality of a representation of one or more objects based on the image data 110. In particular, the image processing system 102 may identify a particular object based on the particular image capture setting and provide a notification as to the quality of the particular object. For example, when the particular image capture setting is the portrait setting, the image processing system 102 may identify a person (e.g., a face) based on the image data 110 and initiate a notification that indicates a quality of the representation of the person within the captured image. To illustrate, the notification may indicate that the image of the person is a high quality image.”
To measure the quality of the image being captured, the criteria for quality must be set beforehand and indicated so that a notification can be made if the standard is met.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the indication of quality standards for the images as taught by Zavesky with the system of Ju in order to show the user the criteria to be met and guide them on how to improve those metrics.
As per claim 8, Ju alone does not explicitly teach the claimed limitations.
However, Ju in combination with Zavesky teaches the claimed:
8. The non-transitory computer readable medium of claim 1, the operations further comprising: outputting a notice of which criteria of the one or more quality criteria are not satisfied and how to satisfy the one or more quality criteria. (Zavesky [0038]: “Responsive to receiving the image data 110, the image processing system 102 may make a determination of a quality of a representation of one or more objects based on the image data 110. In particular, the image processing system 102 may identify a particular object based on the particular image capture setting and provide a notification as to the quality of the particular object. For example, when the particular image capture setting is the portrait setting, the image processing system 102 may identify a person (e.g., a face) based on the image data 110 and initiate a notification that indicates a quality of the representation of the person within the captured image. To illustrate, the notification may indicate that the image of the person is a high quality image. Alternatively, the notification may indicate that the image of the person is poor quality (e.g., blurry or low-lit). Additionally or alternatively, the notification may suggest an alternate image capture setting that may provide a better quality image of the person. In some embodiments, the image processing system 102 may adjust the image capture settings of the camera and prompt a user of the camera to capture another image using the adjusted settings. For example, the image processing system 102 may be configured to automatically adjust the image capture setting or may request permission from the user to adjust the image capture settings.” Zavesky indicates images that are of poor quality that don’t meet certain criteria and recommends ways to satisfy them. This is done to inform future image capture settings.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the indication of a poor-quality image at the time of image capture with a recommendation of new image capture settings as taught by Zavesky with the system of Ju in order to identify the problems with the quality of the images of Ju to allow the user to adjust future capture settings to account for the shortcomings.
As per claim 13, Ju teaches the claimed:
13. The non-transitory computer readable medium of claim 1, the operations further comprising: detecting at least one of motion blur or camera focus associated with the video; and determining at least one of a) that the motion blur fails to satisfy a motion blur criterion or b) that the camera focus fails to satisfy a camera focus criterion. (Ju teaches motion blur in images that fall short of quality standards that are related to image capture settings. Ju [0005]-[0006]: “[0005] Moreover, a camera module with an optical image stabilizer (OIS) is expensive. Hence, the conventional smartphone is generally equipped with a digital image stabilizer (i.e., an electronic image stabilizer (EIS)). The digital image stabilizer can counteract the motion of images, but fails to prevent image blurring.
[0006] In addition to the camera shake, the movement of a target object within a scene to be captured may cause the captured image to have blurry image contents. For example, considering a case where the user wants to use the smartphone to take a picture of a child, the captured image may have a blurry image content of the child if the child is still when the user is going to touch the shutter/capture button and then suddenly moves when the user actually touches the shutter/capture button.”).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Ju in view of Herz and further in view of Zavesky and further in view of Jang and further in view of Lee (Pub No. US 20230401804 A1)
As per claim 5, Ju alone does not explicitly teach the claimed limitations.
However, Ju in combination with Lee teaches the claimed:
5. The non-transitory computer readable medium of claim 4, wherein the replacement frame is generated by inputting the first frame and the second frame into a generative model that generates the replacement frame (Lee [0122]: “According to an embodiment, the image processor 320 may generate the second image in which the teeth area included in the first image is replaced with the virtual teeth area corresponding to the teeth area included in the first image, the virtual teeth area included in the additional image where the virtual teeth template is aligned.
[0136] Hereinafter, for convenience of descriptions, a model generated as the data processing device changes the 3D oral model 410 will be referred to as the virtual oral model 420. The virtual oral model 420 is generated from the 3D oral model 410 and may denote a future target virtual model when dental treatment is performed on the oral cavity of the patient.
[0190] According to an embodiment, the application data 921 may be a second image in which a teeth area of a first image is replaced with a virtual teeth area, and the result data 923 output from the neural network model 922 may be a third image including a face having a different attribute from a face included in the second image.” Lee teaches using a neural network to generate replacement for dental images whose teeth are not aligned as desired. This can include generating a whole new image. This can be used to replace the frames removed for low quality as taught by Ju, instead of using interpolation.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the generation of replacement image frames by a generative model as taught by Lee with the system of Ju in order to have a more rigorous system of interpolating images removed for quality reasons.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Ju in view of Herz and further in view of Zavesky and further in view of Jang and further in view of Hare et al. (Pub No. US 20200066037 A1)
As per claim 9, Ju alone does not explicitly teach the claimed limitations.
However, Ju in combination with Hare teaches the claimed:
9. The non-transitory computer readable medium of claim 1, wherein determining that the video fails to satisfy the one or more quality criteria and providing the guidance are performed during the capturing of the video. (Please see Hare in figures 4a and 5-7. For example, in figure 5 of Hare, the system determines that the video fails to satisfy the one or more quality criteria, e.g. the user’s head is not in the correct place during the capturing of the video. In response, Hare provides guidance (the message “Move Your Face Closer” in figure 5 or “Slowly Turn Your Head to the Right” in figure 7) during the capturing of the video. Also, please see Hare in [0070] “As shown in FIGS. 4A, 5, and 6, where the detection component 220 detects a face, the notification component 230 may initially generate movement elements including movement instructions directing movements to position the face within the visible framing element. For example, the notification component 230 may initially generate a first movement element including an instruction to “Move your face closer.” as shown in FIG. 5. Based on the detection component 220 detecting movement of the face within the field of view but without properly positioning the face for scanning and modeling, the notification component 230 may alternate from FIG. 5 to FIG. 4A, changing the first movement element and the movement instruction to a second movement element. As shown in FIG. 4A, the second movement element includes an instruction to ‘Fit face inside area below.’” In this instance, the detection component 220 in Hare corresponds to the camera capturing video of the user’s face. Also, please see Hare in [0054] as well).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to determining that the video fails to satisfy the one or more quality criteria and providing the guidance are performed during the capturing of the video as taught by Hare with the system of Ju in order to help guide the user to correctly position their face during the video scanning process (Hare in [0133]).
Claims 11 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Ju in view of Herz and further in view of Zavesky and further in view of Jang and further in view of Sachs (Pub No. US 20190122411 A1)
As per claim 11, Ju alone does not explicitly teach the claimed limitations.
However, Ju in combination with Sachs teaches the claimed:
11. The non-transitory computer readable medium of claim 1, the operations further comprising: determining facial landmarks of the face in one or more frames of the video;
determining at least one of a head position, a head orientation, a face angle, or a jaw position based on the facial landmarks;
and determining at least one of a) that the head position fails to satisfy a head position criterion, (Sachs [0141]: “In accordance with some embodiments, the rig parameters are nonlinearly related to a rigged models shape changes. The relationship is nonlinear because nonlinear corrective shapes account for interactions between blend shapes. As such, some groups of rig parameters are mutually exclusive in order to provide plausible animation of the model. Thus, processes for determining the rig parameters in accordance with some embodiments are nonlinear guided optimizations of the parameters. In accordance with many embodiments, the determining of the rig parameters may be performed in stages over different subsets of rig parameters where each subset explains a different fraction of variation in the image. The stages in accordance with several embodiments may include, but are not limited to a rigid solve stage, a mouth solve stage, and a upper face solve stage. The rigid solve stage may determine rig parameters explaining motion of non-deforming head features as a rigid motion of the head and may be used to stabilize the geometry for other stages. The mouth solve stage may determine rig parameters for the jaw opening to match a chin position and for mouth shape networks including, but not limited to, moving of mouth corners inward and/or outwards, and rolling of the lips. In accordance to a few embodiments, complementary or exclusive shape groups may be coupled using optimizer constraints. The upper face solve stage determines the rig parameters for facial features independently of the mouth shape networks. Example of facial features that may have movement determined in the upper face so
lve stage include, but are not limited to, eyes, eyebrows, and nostrils.” The features tracked and used as rig parameters are the facial landmarks. Sachs teaches identifying better head movement that can be used for stages of solving positioning for a 3D model that accurately represents the features or desired features in an image. This determines a desired face position for the head, which could be combined with Ju and Zavesky to determine if an image satisfied the right head position shown, or if the capture of the image needs to be modified.).
b) that the head orientation fails to satisfy a head orientation criterion, c) that the face angle fails to satisfy a face angle criterion, or d) that the jaw position fails to satisfy a jaw position criterion.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the determination of head position in an image as taught by Sachs with the system of Ju in order to use the head position as a quality criterion to determine if the images used for a medical treatment show the subject, the head, in the right way.
As per claim 14, Ju alone does not explicitly teach the claimed limitations.
However, Ju in combination with Sachs teaches the claimed:
14. The non-transitory computer readable medium of claim 1, the operations further comprising: determining an amount of visible teeth in the video; and determining whether the amount of visible teeth satisfies an amount of visible teeth criterion. (Sachs [0122]: “… In FIGS. 31-35, the images 3101 show the facial landmarks 3102 identified by a conventional landmark identification process and the images 3505 show the landmarks 3110 identified by a MDM in accordance with an embodiment of the invention. In FIG. 31, the landmarks 3102 in the image 3101 do not properly align with one side of the face while the landmarks 3110 in the image 3105 align with the side of the face. In FIG. 32, the landmarks 3102 in the image 3101 do not properly align with the mouth and instead are aligned with a portion of the beard while the landmarks 3110 in the image 3105 align with the mouth. In FIG. 33, the landmarks 3102 in the image 3101 do not properly align with the jawline of the face and instead align with the beard while the landmarks 3110 in the image 3105 align with jawline in the beard. In FIG. 34, the landmarks 3102 in the image 3101 do not properly align with the mouth and instead align with the teeth while the landmarks 3110 in the image 3105 align with the lips of the mouth. In FIG. 35, the landmarks 3102 in the image 3101 do not properly align with the mouth and instead aligning with the tongue inside the mouth while the landmarks 3110 in the image 3105 align with properly with the lips of the mouth.” The position of the lips and tongue relative to the teeth will indicate the number of teeth visible. These determine if the image is properly up to a standard for the subject of the image.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the identification of proper alignment of the mouth and features of the mouth as taught by Sachs with the system of Ju and Zavesky in order to give the system of Ju criteria for image quality settings that depend on the images’ depiction of dental features.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Ju in view of Herz and further in view of Zavesky and further in view of Jang and further in view of Sachs and further in view of Wu (Pub No. US 20140371599 A1).
As per claim 12, Ju alone does not explicitly teach the claimed limitations.
However, Ju in combination with Sachs teaches the claimed:
12. The non-transitory computer readable medium of claim 1, the operations further comprising:
determining an optical flow between two or more frames of the video; (Sachs [0079]: “In accordance with some embodiments, a rig is generated for the static 3D model. The rig can be generated by applying a standard set FACS blend shapes to a mesh of the static 3D model of the head. The motion of one or more landmarks and/or 3D shapes in visible video can be tracked and the blend shapes of the static 3D model video recomputed based on the tracked landmarks and/or to provide a customized rig for the 3D model” The tracking, blending, and computation of the motion of the objects in the video is the optical flow.).
determining at least one of a head movement speed or a camera stability based on the optical flow; (The motion of the objects in Sachs includes the head, as describes above in [0141].).
Ju and Sachs alone do not explicitly teach the claimed limitations.
However, Ju in combination with Sachs and Wu teaches the claimed:
and determining at least one of a) that the camera stability fails to satisfy a camera stability criterion or b) that the head movement speed fails to satisfy a head movement speed criterion. (Wu [0056]: “As described herein, one or more processors of a computing device may be configured to identify patient behaviors from video information captured by camera 26. For example, the computing device may be configured to obtaining video information of patient motion captured over a period of time, such that the video information comprises a plurality of frames. The computing device may then receive, with respect to one or more frames of the plurality of frames, a selection of a sample area representative of an anatomical region (e.g., head 14, torso 16, arm 18A, or arm 18B). This sample area may be defined by user input and/or the one or more processors. The computing device may also analyze each of the other plurality of frames for respective areas corresponding to the sample area. The computing device can then calculate one or more movement parameters (e.g., velocity, angle of movement, or frequency of movement) of the anatomical region during the period of time from at least one difference between the sample area and one or more respective areas of at least a subset of the plurality of frames. The computing device may also be configured to compare the one or more movement parameters of the period of time to respective criteria for each of a plurality of predetermined patient behaviors (e.g., types of movements or movement disorders) and identify, based on the comparison, each one of the predetermined patient behaviors that occurred during the period of time.” Wu teaches comparing movement of a patient’s body parts. in an image to predetermined standards. These movements include velocity of a head area.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the tracking of landmarks in a video as taught by Sachs with the system of Ju in order to track the features of the head used to determine if an image meets a quality criterion.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the tracking of the speed of the head in a video as taught by Wu with the system of Ju modified by Sachs in order to track head speed as a criterion for images to determine if they meet a criterion of quality.
Claims 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Ju in view of Zavesky and further in view of Xue (US 20180360567 A1).
As per claim 15, Ju alone does not explicitly teach the claimed limitations.
However, Ju in combination with Xue teaches the claimed:
15. The non-transitory computer readable medium of claim 1, wherein the updated video comprises a current condition of a dental site of the individual, the operations further comprising:
estimating a future condition of the dental site; and modifying the updated video by replacing the current condition of the dental site with the future condition of the dental site in the updated video. (Xue [0074]: “The apparatuses and/or methods (e.g., systems, devices, etc.) described below can be used with and/or integrated into an orthodontic treatment plan. The apparatuses and/or methods described herein may be used to segment a patient's teeth from a two-dimensional image and this segmentation information may be used to simulate, modify and/or choose between various orthodontic treatment plans. Segmenting the patient's teeth can be done automatically (e.g., using a computing device). For example, segmentation can be performed by a computing system automatically by evaluating data (such as three-dimensional scan, or a dental impression) of the patient's teeth or arch.” The treatment plan is the estimated future condition. Xue [0075]: “As described herein, an intraoral scanner may image a patient's dental arch and generate a virtual three-dimensional model of that dental arch. During an intraoral scan procedure (also referred to as a scan session), a user (e.g., a dental practitioner) of an intraoral scanner may generate multiple different images (also referred to as scans or medical images) of a dental site, model of a dental site, or other object. The images may be discrete images (e.g., point-and-shoot images) or frames from a video (e.g., a continuous scan). The three-dimensional scan can generate a 3D mesh of points representing the patient's arch, including the patient's teeth and gums. Further computer processing can segment or separate the 3D mesh of points into individual teeth and gums.” The future treatment plan can be applied to the updated video after the quality criteria have been applied.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use the modeling of an orthodontic treatment as taught by Xue with the system of Ju in order to apply simulate an orthodontic treatment on a video after it has been modified to meet quality criteria.
As per claim 16, Ju teaches the claimed:
16. The non-transitory computer readable medium of claim 15, the operations further comprising: determining one or more quality metric values for a plurality of frames of the modified updated video; identifying two or more consecutive frames of the plurality of frames having one or more quality metric values that fail to satisfy one or more quality metric criteria; and removing the two or more consecutive frames from the modified updated video. (Ju [0043]: “Regarding the aforementioned video recording examples, the video frames are selected based on the image quality metric values. However, if there are too many consecutive video frames dropped due to worse quality, the temporal smoothness of the selected video frames would be significantly degraded. … However, this is not meant to be a limitation of the present invention.” Ju teaches that the replacement of consecutive frames that fail to meet quality standards is possible, even if it is not the ideal implementation of the invention.).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THOMAS JOHN FOSTER whose telephone number is (571)272-5053. The examiner can normally be reached Mon, Fri 8:30-6. Tues-Thurs 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 571-272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THOMAS JOHN FOSTER/Examiner, Art Unit 2616
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights