DETAILED ACTION
This Action is in response to Applicant’s response filed on 11/06/2025. Claims 1-25 are still pending in the present application. This Action is made FINAL.
Response to Amendment
Claim Objection: The amended claims filed on 11/06/2025 overcomes the Claim Objection in the previous office action.
Response to Arguments
Applicant's arguments filed on 11/06/2025 have been fully considered but are moot in view of the new ground(s) rejection in view of Lv et al (U.S. 20130286161 A1; Lv).
Claims Status
Claim(s) 1-3, 6-8, 10-14, 17-19 and 21-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fonte et al (U.S. 20200285081 A1; Fonte), in view of Lv et al (U.S. 20130286161 A1; Lv).
Claim(s) 4-5, 15-16 and 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fonte et al (U.S. 20200285081 A1; Fonte), in view of Lv et al (U.S. 20130286161 A1; Lv), and in further view of Znamenskiy et al (U.S. 20150262422 A1; Znamenskiy).
Claim(s) 9, 20 and 24-25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fonte et al (U.S. 20200285081 A1; Fonte), in view of Lv et al (U.S. 20130286161 A1; Lv), and in further view of Lucey et al (U.S. 20170173289 A1; Lucey).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 6-8, 10-14, 17-19 and 21-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fonte et al (U.S. 20200285081 A1; Fonte), in view of Lv et al (U.S. 20130286161 A1; Lv).
Regarding claim 1, Fonte discloses a computer-implemented method,(Fig.2 and Paragraph 40: “systems are disclosed for creating a custom product. … a processor configured to execute the instructions to perform the method”) comprising:
detecting a feature in the first image data that includes a head of a user, the first image data captured by a computing device at a first position and orientation; (Figs.7-8 and Paragraphs 126-127: “the user has a handheld computer system and moves the camera around their head rather than rotating their head. … The captured video may consist of a series of images of the user's face at various angles making up a set of image data. The computer system may perform analysis on the images immediately as they are captured to provide feedback to the user if there is a problem or if insufficient image quality, poses, or quantity of data is acquired”; Paragraph 133: “The computer system analyzes the image data to iteratively perform a sequence of feature detection, pose estimation, alignment, and model parameter adjustment. A face detection and pose estimation algorithm is used to determine a general position and direction the face is pointing toward, which aids in model position and alignment.”)
detecting the feature in second image data that includes the head of the user, the second image data captured by the computing device at a second position and orientation following a relative lateral movement between the computing device and the head of the user; (Fig.7 Paragraph 126: “the computer system instructs the user to position and move their head while the camera captures a series of images, or video … the user has a handheld computer system and moves the camera around their head rather than rotating their head.”; (Paragraph 120: “A first reference image is captured with a reference object held by the user in the same field as their face. The image data captured by the computer is analyzed by the computer system to detect the reference object and measure its size, for example in pixels. The image data is further analyzed by the computer system to detect one or more of a plurality of features, including but not limited to pupils, eyes, nose, mouth, ears, face, eyebrows, hair, etc.”)
correlating a difference between the first position and the second position of the computing device (Paragraph 139-143: “the center of the eyes, corners of the eyes, tip of the nose, top of the ears, and other important landmarks is detected and tracked through multiple images. … the model coordinates and camera position align the face model with the pose, position, and scale of the images of the user's face 1703”) with a change in the position of the feature between the first and second image (Paragraphs 139-140: “the center of the eyes, corners of the eyes, tip of the nose, top of the ears, and other important landmarks is detected and tracked through multiple images. These simple points, oriented in space in a dataset, provide all the information needed to obtain quantitative information needed for subsequent analyses. … the use of out of focus areas or the parallax between adjacent images is used to estimate depth.”) data; (Paragraph 153: “c) During acquisition of the close-up image, the user positions the computer system to obtain an image of at least some facial features, while the computer system also measures distance from the computer to the user; d) The computer system detects facial features (iris, pupil, etc) in the close-up image and measure the distance between the features; e) The computer system uses the distance measured from the computer to the user and intrinsic camera properties to determine the scale of pixels in the image data; f) The computer system determines reference distances between facial features based on the image scale and measured distance between features;”)
determining a scale value applicable to the first image data and the second image data based on the correlating; (Paragraph 139-143: “the center of the eyes, corners of the eyes, tip of the nose, top of the ears, and other important landmarks is detected and tracked through multiple images. … the model coordinates and camera position align the face model with the pose, position, and scale of the images of the user's face 1703”; (Paragraph 153: “c) During acquisition of the close-up image, the user positions the computer system to obtain an image of at least some facial features, while the computer system also measures distance from the computer to the user; d) The computer system detects facial features (iris, pupil, etc) in the close-up image and measure the distance between the features; e) The computer system uses the distance measured from the computer to the user and intrinsic camera properties to determine the scale of pixels in the image data; f) The computer system determines reference distances between facial features based on the image scale and measured distance between features;”) and
performing a virtual selection of a wearable device using the scale value. (Paragraph 120-121: “With the data previously analyzed from the reference object the distance in pixels between pupils or other features is scaled from pixels to a unit of distance such as millimeters or inches … The purpose of scaling the data with a reference object is to ensure that measurements can be derived from the final quantitative anatomic model of the user. There are several key measurements to best determine how to virtually place and fit eyewear on an image of a user's face”; Paragraph 180-181; Paragraph 222: “systems and methods for creating previews of custom eyewear on the user's image or anatomic data. The quantitative anatomic model of the user's face is established, scaled, and registered to the image data such that the model coordinates and camera position align the face model with the pose, position, and zoom level of the images of the user's face.”)
However, Fonte does not disclose correlating a difference between the first orientation and the second orientation of the computing device with a change in the position of the feature between the first and second image data
Lv discloses detecting a feature (Fig.5a: feature point 510.1) in first image data (Fig.5a: image 506.1) that includes a head of a user, the first image data captured by a computing device at a first position (Fig.5a: position 504.1) and orientation (Fig.5a: orientation data 508.1) ; detecting the feature (Fig.5a: feature point 510.2) in second image data that includes the head of the user, the second image data (Fig.5a: image 506.2) captured by the computing device at a second position (Fig.5a: position 504.2) and orientation (Fig.5a: orientation data 508.2) following a relative lateral movement between the computing device and the head of the user; (Figs. 1-2; Figs. 5A-5B and Paragraph 35: “the device captures a set of images of the local user's face (operation 204), and processes the captured images to detect facial features of the local user (operation 206). The device then determines orientation information for each captured image (operation 208) .Generates the three-dimensional model of the user's face from the orientation information for the captured images and the image coordinates of the detected features (operation 210). … The device analyzes these captured images to detect position information on the images for certain facial features, and uses the device motion or orientation information to efficiently compute the 3-D position of these features” ;Paragraphs 49-51)
correlating a difference between the first position and orientation and the second position and orientation of the computing device with a change in the position of the feature between the first and second image data; (Paragraph 49-51: “When the user begins the image-capture operation, the image-capture device captures image 506.1 and orientation data 508.1 while the device is in orientation 504.1. As the user sweeps the device in front of his/her face, the device can capture images 506.2 through 506.j and orientation data 508.2 through 508.j for device orientations 504.2 through 504.j, respectively … the device can determine feature points 510.1, 510.2, and 510.j that correspond to a nose feature captured by images 506.1, 506.2, and 506.j, respectively. The coordinates of a feature point i within an image j is hereinafter denoted using the tuple (u.sub.i.sup.(j), v.sub.i.sup.(j)). The device then processes the orientation data 508 and the feature points 510 to generate a three-dimensional model in a global coordinate system.”; Paragraphs 57-61: “the system of linear equations includes a linear equation for each feature point of each captured image (e.g., for a feature point i from an image view j, .. These equations map the coordinates of these feature points from the global coordinate system using a transformation … The device uses the gyroscope data to compute an accurate rotation matrix R.sub.j,0, which facilitates generating the three-dimensional model by solving the set of linear equations, … input the rotation matrix computed from gyroscope data, which corresponds to a rotation of the device from view 0 to view j. The tuple (u.sub.i.sup.(0), v.sub.i.sup.(0)) takes as input an image coordinate detected for the facial marker i from the image captured from view 0, and the tuple (u.sub.u.sup.(j), v.sub.i.sup.(j)) takes as input an image coordinate detected for the facial marker i from the image captured from view j”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte by including generates a corresponding three-dimensional facial model for the user that is taught by Lv, to make the invention that using a mobile device that includes an image sensor and a motion sensor to generate a three-dimensional model of a user's face; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the detected of facial marker in all views and provides a solution that is robust toward errors in detecting feature coordinates from the individual views. (Lv: Paragraph 66)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 2, Fonte, as modified by Lv discloses all the claims invention. Fonte further discloses the feature includes at least one facial feature. (Paragraph 120: “The image data is further analyzed by the computer system to detect one or more of a plurality of features, including but not limited to pupils, eyes, nose, mouth, ears, face, eyebrows, hair, etc.”)
Regarding claim 3, Fonte, as modified by Lv discloses all the claims invention. Fonte further discloses the at least one facial feature includes at least one of: a distance between an outer corner portion of a right eye and an outer corner portion of a left eye of the user; a distance between an inner corner portion of a right eye and an inner corner portion of a left eye of the user; or a distance between a pupil of a right eye and a pupil of a left eye of the user. (Fig.7 and Paragraph 125: “ FIG. 7 shows eyewear 701 with binocular interpupillary distance (Pd) 703a between pupils 702; Paragraphs 150-151: “The computer system detects eye features of the user (pupils, irises, etc) in the face model and measure the distance between the eye features; … the computer system detects facial features (pupils, irises, eye corners, mouth corners, nose, etc) in at least one image and measure the un-scaled distance between them;”)
Regarding claim 6, Fonte, as modified by Lv discloses all the claims invention. Lv further discloses detecting the change in the position and the orientation of the computing device includes: detecting the first position (Fig.5a: position 504.1) and the first orientation (Fig.5a: orientation data 508.1) of the computing device in response to receiving first data provided by an inertial measurement unit of the computing device at the capturing of the first image data; (Paragraphs 26: “The device can also use the on-board motion sensor and its face-detection capabilities to determine the right moments to capture an image as the user sweeps the device in front of his/her face, and can inform the user if the user is performing the sweeping motion incorrectly.”; Paragraph 50: “The image-capture device can determine orientation data 508 using any motion-sensor, now known or later developed, that can determine absolute or relative three-dimensional coordinates for each captured image. For example, the motion sensor can include a gyroscope that provides three rotation angles perpendicular to the device's plane (e.g., the pitch, yaw, and roll angles along the X, Y, and Z axis, respectively) for each captured image.”)
detecting the second position (Fig.5a: position 504.2) and the second orientation(Fig.5a: orientation data 508.2)of the computing device in response to receiving second data provided by the inertial measurement unit of the computing device at the capturing of the second image data; Paragraphs 26: “The device can also use the on-board motion sensor and its face-detection capabilities to determine the right moments to capture an image as the user sweeps the device in front of his/her face, and can inform the user if the user is performing the sweeping motion incorrectly.”; Paragraph 50: “The image-capture device can determine orientation data 508 using any motion-sensor, now known or later developed, that can determine absolute or relative three-dimensional coordinates for each captured image. For example, the motion sensor can include a gyroscope that provides three rotation angles perpendicular to the device's plane (e.g., the pitch, yaw, and roll angles along the X, Y, and Z axis, respectively) for each captured image.”) and
determining a magnitude of movement of the computing device corresponding to the change in the position and the orientation of the computing device based on a comparison of the second data to the first data. (Paragraph 27: “the device analyzes these captured images to detect position information on the images for certain facial features, and uses the device motion or orientation information to efficiently compute the 3-D position of these features and generates a corresponding three-dimensional facial model for the user. Once the device generates the three-dimensional model, the device can normalize the scale and orientation of the model with respect to a global coordinate system,”; Paragraphs 47: “the device monitors a change in its orientation from that of a previous stored image (operation 416), and determines whether the orientation has changed by at least a minimum threshold (operation 418).”; Paragraph 61: “The 3.times.3 matrix R.sub.j,0 takes as input the rotation matrix computed from gyroscope data, which corresponds to a rotation of the device from view 0 to view j. The tuple (u.sub.i.sup.(0), v.sub.i.sup.(0)) takes as input an image coordinate detected for the facial marker i from the image captured from view 0, and the tuple (u.sub.u.sup.(j), v.sub.i.sup.(j)) takes as input an image coordinate detected for the facial marker i from the image captured from view j.”)
Regarding claim 7, Fonte, as modified by Lv discloses all the claims invention. Lv further discloses correlating the change in the position and the orientation of the computing device with the change in the position of the feature includes: associating the magnitude of the movement of the computing device to the change in the position of the feature; (Paragraph 49-50: “Fig.G. 5A illustrates a motion trajectory 500 of an image-capture device 502 during an image capture operation … the motion sensor can include a gyroscope that provides three rotation angles perpendicular to the device's plane (e.g., the pitch, yaw, and roll angles along the X, Y, and Z axis, respectively) for each captured image.”; Paragraph 61”) and assigning a scale value based on the associating. (Paragraphs 26-27: “The device can also use the on-board motion sensor and its face-detection capabilities to determine the right moments to capture an image as the user sweeps the device in front of his/her face, and can inform the user if the user is performing the sweeping motion incorrectly. … the device analyzes these captured images to detect position information on the images for certain facial features, and uses the device motion or orientation information to efficiently compute the 3-D position of these features and generates a corresponding three-dimensional facial model for the user. Once the device generates the three-dimensional model, the device can normalize the scale and orientation of the model with respect to a global coordinate system,”; Paragraph 42: “the device can monitor the motion using an on-board gyroscope, and can monitor the quality of a captured image by analyzing its brightness, contrast, sharpness, and/or by counting the number of detectable facial features.”; Paragraphs 47: “the device monitors a change in its orientation from that of a previous stored image (operation 416), and determines whether the orientation has changed by at least a minimum threshold (operation 418).”; Paragraph 61”)
Regarding claim 8, Fonte, as modified by Lv discloses all the claims invention. Fonte further discloses capturing the first image data and capturing the second image data includes: initiating operation of a front facing camera of the computing device; (Fig.5 and Paragraph 120: “ FIG. 5 shows a user 501 using a computer device 502 to acquire image data of their face 503. Instructions are provided to the user to place their face in certain positions while the computer system captures and analyzes image data of the user's face. The computer system may utilize a smart phone or handheld electronic camera for the capture of the image of the person's face.”) and capturing, by the front facing camera, the first image data and the second image data as the computing device is moved relative to the head of the user. (Paragraphs 128-129: “the user has a handheld computer system and moves the camera around their head rather than rotating their head. … The captured video may consist of a series of images of the user's face at various angles making up a set of image data.”)
Regarding claim 10, Fonte, as modified by Lv discloses all the claims invention. Lv discloses further discloses the capturing first image data and the capturing the second image data includes sequentially capturing a first image and a second image. (Figs. 1-2; Paragraph 31: “When device 102 is ready to generate the three-dimensional model, device 102 instructs user 104 to sweep device 102 across his/her face to capture his/her face from various positions and orientations (e.g., positions 106.1, 106.2 and 106.j). … The image-capturing procedure is continuous and automatic,”)
Regarding claim 11, Fonte, as modified by Lv discloses all the claims invention. Lucey further discloses the capturing the first image data and the capturing the second image data includes capturing additional image data between the capturing of the first image data and the second image data. ( “Figs 5A-5B: image 506.1; image 506.2 and image 506.j; Paragraph 49: “the user sweeps the device in front of his/her face, the device can capture images 506.1, 506.2 through 506.j and orientation data 508.2 through 508.j for device orientations 504.2 through 504.j, respectively.”)
Regarding claim 12, Fonte discloses a non-transitory computer-readable medium storing executable instructions that when executed by at least one processor of a computing device are configured to cause the at least one processor(Fig.2 and Paragraph 40: “systems are disclosed for creating a custom product … a digital storage device to store instructions for creating and previewing custom product; a processor configured to execute the instructions to perform the method”) to:
detect a feature in the first image data that includes a head of a user, the first image data captured by a computing device at a first position and orientation; (Figs.7-8 and Paragraphs 126-127: “the user has a handheld computer system and moves the camera around their head rather than rotating their head. … The captured video may consist of a series of images of the user's face at various angles making up a set of image data. The computer system may perform analysis on the images immediately as they are captured to provide feedback to the user if there is a problem or if insufficient image quality, poses, or quantity of data is acquired”; Paragraph 133: “The computer system analyzes the image data to iteratively perform a sequence of feature detection, pose estimation, alignment, and model parameter adjustment. A face detection and pose estimation algorithm is used to determine a general position and direction the face is pointing toward, which aids in model position and alignment.”)
detect the feature in second image data that includes the head of the user, the second image data captured by the computing device at a second position and orientation following a relative lateral movement between the computing device and the head of the user; (Fig.7 Paragraph 126: “the computer system instructs the user to position and move their head while the camera captures a series of images, or video … the user has a handheld computer system and moves the camera around their head rather than rotating their head.”; (Paragraph 120: “A first reference image is captured with a reference object held by the user in the same field as their face. The image data captured by the computer is analyzed by the computer system to detect the reference object and measure its size, for example in pixels. The image data is further analyzed by the computer system to detect one or more of a plurality of features, including but not limited to pupils, eyes, nose, mouth, ears, face, eyebrows, hair, etc.”)
correlate a difference between the first position and the second position of the computing device (Paragraph 139-143: “the center of the eyes, corners of the eyes, tip of the nose, top of the ears, and other important landmarks is detected and tracked through multiple images. … the model coordinates and camera position align the face model with the pose, position, and scale of the images of the user's face 1703”) with a change in the position of the feature between the first and second image (Paragraphs 139-140: “the center of the eyes, corners of the eyes, tip of the nose, top of the ears, and other important landmarks is detected and tracked through multiple images. These simple points, oriented in space in a dataset, provide all the information needed to obtain quantitative information needed for subsequent analyses. … the use of out of focus areas or the parallax between adjacent images is used to estimate depth.”) data; (Paragraph 153: “c) During acquisition of the close-up image, the user positions the computer system to obtain an image of at least some facial features, while the computer system also measures distance from the computer to the user; d) The computer system detects facial features (iris, pupil, etc) in the close-up image and measure the distance between the features; e) The computer system uses the distance measured from the computer to the user and intrinsic camera properties to determine the scale of pixels in the image data; f) The computer system determines reference distances between facial features based on the image scale and measured distance between features;”)
determine a scale value applicable to the first image data and the second image data based on the correlation;(Paragraph 139-143: “the center of the eyes, corners of the eyes, tip of the nose, top of the ears, and other important landmarks is detected and tracked through multiple images. … the model coordinates and camera position align the face model with the pose, position, and scale of the images of the user's face 1703”; (Paragraph 153: “c) During acquisition of the close-up image, the user positions the computer system to obtain an image of at least some facial features, while the computer system also measures distance from the computer to the user; d) The computer system detects facial features (iris, pupil, etc) in the close-up image and measure the distance between the features; e) The computer system uses the distance measured from the computer to the user and intrinsic camera properties to determine the scale of pixels in the image data; f) The computer system determines reference distances between facial features based on the image scale and measured distance between features;”) and
perform a virtual selection of a wearable device using the scale value. (Paragraph 120-121: “With the data previously analyzed from the reference object the distance in pixels between pupils or other features is scaled from pixels to a unit of distance such as millimeters or inches … The purpose of scaling the data with a reference object is to ensure that measurements can be derived from the final quantitative anatomic model of the user. There are several key measurements to best determine how to virtually place and fit eyewear on an image of a user's face”; Paragraph 180-181; Paragraph 222: “systems and methods for creating previews of custom eyewear on the user's image or anatomic data. The quantitative anatomic model of the user's face is established, scaled, and registered to the image data such that the model coordinates and camera position align the face model with the pose, position, and zoom level of the images of the user's face.”)
However, Fonte does not disclose correlate a difference between the first orientation and the second orientation of the computing device with a change in the position of the feature between the first and second image data
Lv discloses detecting a feature (Fig.5a: feature point 510.1) in first image data (Fig.5a: image 506.1) that includes a head of a user, the first image data captured by a computing device at a first position (Fig.5a: position 504.1) and orientation (Fig.5a: orientation data 508.1) ; detecting the feature (Fig.5a: feature point 510.2) in second image data that includes the head of the user, the second image data (Fig.5a: image 506.2) captured by the computing device at a second position (Fig.5a: position 504.2) and orientation (Fig.5a: orientation data 508.2) following a relative lateral movement between the computing device and the head of the user; (Figs. 1-2; Figs. 5A-5B and Paragraph 35: “the device captures a set of images of the local user's face (operation 204), and processes the captured images to detect facial features of the local user (operation 206). The device then determines orientation information for each captured image (operation 208) .Generates the three-dimensional model of the user's face from the orientation information for the captured images and the image coordinates of the detected features (operation 210). … The device analyzes these captured images to detect position information on the images for certain facial features, and uses the device motion or orientation information to efficiently compute the 3-D position of these features” ;Paragraphs 49-51)
correlating a difference between the first position and orientation and the second position and orientation of the computing device with a change in the position of the feature between the first and second image data; (Paragraph 49-51: “When the user begins the image-capture operation, the image-capture device captures image 506.1 and orientation data 508.1 while the device is in orientation 504.1. As the user sweeps the device in front of his/her face, the device can capture images 506.2 through 506.j and orientation data 508.2 through 508.j for device orientations 504.2 through 504.j, respectively … the device can determine feature points 510.1, 510.2, and 510.j that correspond to a nose feature captured by images 506.1, 506.2, and 506.j, respectively. The coordinates of a feature point i within an image j is hereinafter denoted using the tuple (u.sub.i.sup.(j), v.sub.i.sup.(j)). The device then processes the orientation data 508 and the feature points 510 to generate a three-dimensional model in a global coordinate system.”; Paragraphs 57-61: “the system of linear equations includes a linear equation for each feature point of each captured image (e.g., for a feature point i from an image view j, .. These equations map the coordinates of these feature points from the global coordinate system using a transformation … The device uses the gyroscope data to compute an accurate rotation matrix R.sub.j,0, which facilitates generating the three-dimensional model by solving the set of linear equations, … input the rotation matrix computed from gyroscope data, which corresponds to a rotation of the device from view 0 to view j. The tuple (u.sub.i.sup.(0), v.sub.i.sup.(0)) takes as input an image coordinate detected for the facial marker i from the image captured from view 0, and the tuple (u.sub.u.sup.(j), v.sub.i.sup.(j)) takes as input an image coordinate detected for the facial marker i from the image captured from view j”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte by including generates a corresponding three-dimensional facial model for the user that is taught by Lv, to make the invention that using a mobile device that includes an image sensor and a motion sensor to generate a three-dimensional model of a user's face; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the detected of facial marker in all views and provides a solution that is robust toward errors in detecting feature coordinates from the individual views. (Lv: Paragraph 66)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 13, Fonte, as modified by Lv discloses all the claims invention. Fonte further discloses the feature includes at least one facial feature. (Paragraph 120: “The image data is further analyzed by the computer system to detect one or more of a plurality of features, including but not limited to pupils, eyes, nose, mouth, ears, face, eyebrows, hair, etc.”)
Regarding claim 14, Fonte, as modified by Lv discloses all the claims invention. Fonte further discloses the at least one facial feature includes at least one of: a distance between an outer corner portion of a right eye and an outer corner portion of a left eye of the user; a distance between an inner corner portion of a right eye and an inner corner portion of a left eye of the user; or a distance between a pupil of a right eye and a pupil of a left eye of the user. (Fig.7 and Paragraph 125: “ FIG. 7 shows eyewear 701 with binocular interpupillary distance (Pd) 703a between pupils 702; Paragraphs 150-151: “The computer system detects eye features of the user (pupils, irises, etc) in the face model and measure the distance between the eye features; … the computer system detects facial features (pupils, irises, eye corners, mouth corners, nose, etc) in at least one image and measure the un-scaled distance between them;”)
Regarding claim 17, Fonte, as modified by Lv discloses all the claims invention. Lv further discloses: detect the first position(Fig.5a: position 504.1) and the first orientation (Fig.5a: orientation data 508.1) of the computing device in response to receiving first data provided by an inertial measurement unit of the computing device at the capturing of the first image data; (Paragraphs 26: “The device can also use the on-board motion sensor and its face-detection capabilities to determine the right moments to capture an image as the user sweeps the device in front of his/her face, and can inform the user if the user is performing the sweeping motion incorrectly.”; Paragraph 50: “The image-capture device can determine orientation data 508 using any motion-sensor, now known or later developed, that can determine absolute or relative three-dimensional coordinates for each captured image. For example, the motion sensor can include a gyroscope that provides three rotation angles perpendicular to the device's plane (e.g., the pitch, yaw, and roll angles along the X, Y, and Z axis, respectively) for each captured image.”)
detect the second position (Fig.5a: position 504.2) and the second orientation(Fig.5a: orientation data 508.2)of the computing device in response to receiving second data provided by the inertial measurement unit of the computing device at the capturing of the second image data; Paragraphs 26: “The device can also use the on-board motion sensor and its face-detection capabilities to determine the right moments to capture an image as the user sweeps the device in front of his/her face, and can inform the user if the user is performing the sweeping motion incorrectly.”; Paragraph 50: “The image-capture device can determine orientation data 508 using any motion-sensor, now known or later developed, that can determine absolute or relative three-dimensional coordinates for each captured image. For example, the motion sensor can include a gyroscope that provides three rotation angles perpendicular to the device's plane (e.g., the pitch, yaw, and roll angles along the X, Y, and Z axis, respectively) for each captured image.”) and
determine a magnitude of movement of the computing device corresponding to the change in the position and the orientation of the computing device based on a comparison of the second data to the first data. (Paragraph 27: “the device analyzes these captured images to detect position information on the images for certain facial features, and uses the device motion or orientation information to efficiently compute the 3-D position of these features and generates a corresponding three-dimensional facial model for the user. Once the device generates the three-dimensional model, the device can normalize the scale and orientation of the model with respect to a global coordinate system,”; Paragraphs 47: “the device monitors a change in its orientation from that of a previous stored image (operation 416), and determines whether the orientation has changed by at least a minimum threshold (operation 418).”; Paragraph 61: “The 3.times.3 matrix R.sub.j,0 takes as input the rotation matrix computed from gyroscope data, which corresponds to a rotation of the device from view 0 to view j. The tuple (u.sub.i.sup.(0), v.sub.i.sup.(0)) takes as input an image coordinate detected for the facial marker i from the image captured from view 0, and the tuple (u.sub.u.sup.(j), v.sub.i.sup.(j)) takes as input an image coordinate detected for the facial marker i from the image captured from view j.”)
Regarding claim 18, Fonte, as modified by Lv discloses all the claims invention. Lv further discloses associate the magnitude of the movement of the computing device to the change in the position of the feature; ; (Paragraph 49-50: “Fig.G. 5A illustrates a motion trajectory 500 of an image-capture device 502 during an image capture operation … the motion sensor can include a gyroscope that provides three rotation angles perpendicular to the device's plane (e.g., the pitch, yaw, and roll angles along the X, Y, and Z axis, respectively) for each captured image.”; Paragraph 61”) and assign a scale value based on the association of the magnitude of the movement of the computing device with the change in the position of the feature. (Paragraphs 26-27: “The device can also use the on-board motion sensor and its face-detection capabilities to determine the right moments to capture an image as the user sweeps the device in front of his/her face, and can inform the user if the user is performing the sweeping motion incorrectly. … the device analyzes these captured images to detect position information on the images for certain facial features, and uses the device motion or orientation information to efficiently compute the 3-D position of these features and generates a corresponding three-dimensional facial model for the user. Once the device generates the three-dimensional model, the device can normalize the scale and orientation of the model with respect to a global coordinate system,”; Paragraph 42: “the device can monitor the motion using an on-board gyroscope, and can monitor the quality of a captured image by analyzing its brightness, contrast, sharpness, and/or by counting the number of detectable facial features.”; Paragraphs 47: “the device monitors a change in its orientation from that of a previous stored image (operation 416), and determines whether the orientation has changed by at least a minimum threshold (operation 418).”; Paragraph 61”)
Regarding claim 19, Fonte, as modified by Lv discloses all the claims invention. Fonte further discloses initiate operation of a front facing camera of the computing device; (Fig.5 and Paragraph 120: “ FIG. 5 shows a user 501 using a computer device 502 to acquire image data of their face 503. Instructions are provided to the user to place their face in certain positions while the computer system captures and analyzes image data of the user's face. The computer system may utilize a smart phone or handheld electronic camera for the capture of the image of the person's face.”) and capture, by the front facing camera, the first image data and the second image data as the computing device is moved relative to the head of the user. (Paragraphs 128-129: “the user has a handheld computer system and moves the camera around their head rather than rotating their head. … The captured video may consist of a series of images of the user's face at various angles making up a set of image data.”)
Regarding claim 21, Fonte, as modified by Lv discloses all the claims invention. Lv further discloses capture the first image data and the second image data sequentially; (Figs. 1-2; Paragraph 31: “When device 102 is ready to generate the three-dimensional model, device 102 instructs user 104 to sweep device 102 across his/her face to capture his/her face from various positions and orientations (e.g., positions 106.1, 106.2 and 106.j). … The image-capturing procedure is continuous and automatic,”) or capture additional image data between the capture of the first image data and the second image data. ( “Figs 5A-5B: image 506.1; image 506.2 and image 506.j; Paragraph 49: “the user sweeps the device in front of his/her face, the device can capture images 506.1, 506.2 through 506.j and orientation data 508.2 through 508.j for device orientations 504.2 through 504.j, respectively.”)
Regarding claim 22, Fonte discloses a system, comprising a computing device, (Fig.2: computer system 220) including:
an image sensor (Fig.2 Image capture device 210 ; Paragraph 107: “at least one computer system 220, including but not limited to a tablet, phone, desktop, laptop, kiosk, or wearable computer, … an image capture device 210, including but not limited to a single-lens camera, video camera, multi-lens camera, IR camera, laser scanner, interferometer, etc.”);
at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the at least one processor (Fig.2 and Paragraph 40: “systems are disclosed for creating a custom product … a digital storage device to store instructions for creating and previewing custom product; a processor configured to execute the instructions to perform the method”) to:
detect a feature in the first image data that includes a head of a user, the first image data captured by a computing device at a first position and orientation; (Figs.7-8 and Paragraphs 126-127: “the user has a handheld computer system and moves the camera around their head rather than rotating their head. … The captured video may consist of a series of images of the user's face at various angles making up a set of image data. The computer system may perform analysis on the images immediately as they are captured to provide feedback to the user if there is a problem or if insufficient image quality, poses, or quantity of data is acquired”; Paragraph 133: “The computer system analyzes the image data to iteratively perform a sequence of feature detection, pose estimation, alignment, and model parameter adjustment. A face detection and pose estimation algorithm is used to determine a general position and direction the face is pointing toward, which aids in model position and alignment.”)
detect the feature in second image data that includes the head of the user, the second image data captured by the computing device at a second position and orientation following a relative lateral movement between the computing device and the head of the user;; (Fig.7 Paragraph 126: “the computer system instructs the user to position and move their head while the camera captures a series of images, or video … the user has a handheld computer system and moves the camera around their head rather than rotating their head.”; (Paragraph 120: “A first reference image is captured with a reference object held by the user in the same field as their face. The image data captured by the computer is analyzed by the computer system to detect the reference object and measure its size, for example in pixels. The image data is further analyzed by the computer system to detect one or more of a plurality of features, including but not limited to pupils, eyes, nose, mouth, ears, face, eyebrows, hair, etc.”)
correlate a difference between the first position and the second position of the computing device (Paragraph 139-143: “the center of the eyes, corners of the eyes, tip of the nose, top of the ears, and other important landmarks is detected and tracked through multiple images. … the model coordinates and camera position align the face model with the pose, position, and scale of the images of the user's face 1703”) with a change in the position of the feature between the first and second image (Paragraphs 139-140: “the center of the eyes, corners of the eyes, tip of the nose, top of the ears, and other important landmarks is detected and tracked through multiple images. These simple points, oriented in space in a dataset, provide all the information needed to obtain quantitative information needed for subsequent analyses. … the use of out of focus areas or the parallax between adjacent images is used to estimate depth.”) data; (Paragraph 153: “c) During acquisition of the close-up image, the user positions the computer system to obtain an image of at least some facial features, while the computer system also measures distance from the computer to the user; d) The computer system detects facial features (iris, pupil, etc) in the close-up image and measure the distance between the features; e) The computer system uses the distance measured from the computer to the user and intrinsic camera properties to determine the scale of pixels in the image data; f) The computer system determines reference distances between facial features based on the image scale and measured distance between features;”)
determine a scale value applicable to the first image data and the second image data based on the correlating;(Paragraph 139-143: “the center of the eyes, corners of the eyes, tip of the nose, top of the ears, and other important landmarks is detected and tracked through multiple images. … the model coordinates and camera position align the face model with the pose, position, and scale of the images of the user's face 1703”; (Paragraph 153: “c) During acquisition of the close-up image, the user positions the computer system to obtain an image of at least some facial features, while the computer system also measures distance from the computer to the user; d) The computer system detects facial features (iris, pupil, etc) in the close-up image and measure the distance between the features; e) The computer system uses the distance measured from the computer to the user and intrinsic camera properties to determine the scale of pixels in the image data; f) The computer system determines reference distances between facial features based on the image scale and measured distance between features;”) and
perform a virtual selection of a wearable device using the scale value. (Paragraph 120-121: “With the data previously analyzed from the reference object the distance in pixels between pupils or other features is scaled from pixels to a unit of distance such as millimeters or inches … The purpose of scaling the data with a reference object is to ensure that measurements can be derived from the final quantitative anatomic model of the user. There are several key measurements to best determine how to virtually place and fit eyewear on an image of a user's face”; Paragraph 180-181; Paragraph 222: “systems and methods for creating previews of custom eyewear on the user's image or anatomic data. The quantitative anatomic model of the user's face is established, scaled, and registered to the image data such that the model coordinates and camera position align the face model with the pose, position, and zoom level of the images of the user's face.”)
However, Fonte does not disclose correlate a difference between the first orientation and the second orientation of the computing device with a change in the position of the feature between the first and second image data
Lv discloses detecting a feature (Fig.5a: feature point 510.1) in first image data (Fig.5a: image 506.1) that includes a head of a user, the first image data captured by a computing device at a first position (Fig.5a: position 504.1) and orientation (Fig.5a: orientation data 508.1) ; detecting the feature (Fig.5a: feature point 510.2) in second image data that includes the head of the user, the second image data (Fig.5a: image 506.2) captured by the computing device at a second position (Fig.5a: position 504.2) and orientation (Fig.5a: orientation data 508.2) following a relative lateral movement between the computing device and the head of the user; (Figs. 1-2; Figs. 5A-5B and Paragraph 35: “the device captures a set of images of the local user's face (operation 204), and processes the captured images to detect facial features of the local user (operation 206). The device then determines orientation information for each captured image (operation 208) .Generates the three-dimensional model of the user's face from the orientation information for the captured images and the image coordinates of the detected features (operation 210). … The device analyzes these captured images to detect position information on the images for certain facial features, and uses the device motion or orientation information to efficiently compute the 3-D position of these features” ;Paragraphs 49-51)
correlating a difference between the first position and orientation and the second position and orientation of the computing device with a change in the position of the feature between the first and second image data; (Paragraph 49-51: “When the user begins the image-capture operation, the image-capture device captures image 506.1 and orientation data 508.1 while the device is in orientation 504.1. As the user sweeps the device in front of his/her face, the device can capture images 506.2 through 506.j and orientation data 508.2 through 508.j for device orientations 504.2 through 504.j, respectively … the device can determine feature points 510.1, 510.2, and 510.j that correspond to a nose feature captured by images 506.1, 506.2, and 506.j, respectively. The coordinates of a feature point i within an image j is hereinafter denoted using the tuple (u.sub.i.sup.(j), v.sub.i.sup.(j)). The device then processes the orientation data 508 and the feature points 510 to generate a three-dimensional model in a global coordinate system.”; Paragraphs 57-61: “the system of linear equations includes a linear equation for each feature point of each captured image (e.g., for a feature point i from an image view j, .. These equations map the coordinates of these feature points from the global coordinate system using a transformation … The device uses the gyroscope data to compute an accurate rotation matrix R.sub.j,0, which facilitates generating the three-dimensional model by solving the set of linear equations, … input the rotation matrix computed from gyroscope data, which corresponds to a rotation of the device from view 0 to view j. The tuple (u.sub.i.sup.(0), v.sub.i.sup.(0)) takes as input an image coordinate detected for the facial marker i from the image captured from view 0, and the tuple (u.sub.u.sup.(j), v.sub.i.sup.(j)) takes as input an image coordinate detected for the facial marker i from the image captured from view j”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte by including generates a corresponding three-dimensional facial model for the user that is taught by Lv, to make the invention that using a mobile device that includes an image sensor and a motion sensor to generate a three-dimensional model of a user's face; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the detected of facial marker in all views and provides a solution that is robust toward errors in detecting feature coordinates from the individual views. (Lv: Paragraph 66)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Claim(s) 4-5, 15-16 and 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fonte et al (U.S. 20200285081 A1; Fonte), in view of Lv et al (U.S. 20130286161 A1; Lv), and in further view of Znamenskiy et al (U.S. 20150262422 A1; Znamenskiy).
Regarding claim 4, Fonte, as modified by Lv discloses all the claims invention except wherein the feature includes a fixed element detected in a background area surrounding the head of the user.
Znamenskiy discloses the feature includes a fixed element (edges 49) detected in a background area surrounding the head of the user. (Figs. 1-4 and Paragraph 115: “Similar to edges 46, edges 49 can be regarded as fiducial markers as well. Accordingly, a high contrast between the surface 16 and the background, in this embodiment in the opening 18, is beneficial for a reliable and exact detection of the edges 49 in an image of template 13, for example by a computer. Since the background is typically provided by a face of a subject in opening 18, such a contrast is advantageously generally given within this embodiment.”; Paragraph 107;)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte and Lv by including a template for collecting data of a face of a subject that is taught by Znamenskiy, to make the invention that a system for collecting data of a face of a subject; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the quality and detailed measurements in the face of the subject as well as reducing time in image analysis such as a defined segmentation between the face and the background (Znamenskiy: Paragraph 30)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 5, Fonte, as modified by Lv discloses all the claims invention except wherein the feature includes a plurality of features, including: at least one facial landmark defined by two facial features; and at least one element defined by at least two fixed key points detected in a background area surrounding the head of the user.
Znamenskiy discloses the feature includes a plurality of features, including: at least one facial landmark defined by two facial features; ; (Fig.7 and Paragraph 133: “the processing unit 54 may recognize facial landmarks like eyes 88, the nose 74 or the mouth 76 of the subject 70. Therefore, a first face detection step will be applied to the image of template 10 and face 72 of the subject 70.”) and at least one element defined by at least two fixed key points (Fig.4; plurality edges 49; Fig.1 :markings 22, 24, 26 and 28) detected in a background area surrounding the head of the user. (Figs. 1-4 and Paragraph 115: “Similar to edges 46, edges 49 can be regarded as fiducial markers as well. Accordingly, a high contrast between the surface 16 and the background, in this embodiment in the opening 18, is beneficial for a reliable and exact detection of the edges 49 in an image of template 13, for example by a computer. Since the background is typically provided by a face of a subject in opening 18, such a contrast is advantageously generally given within this embodiment.”; Paragraph 125: “The processing unit 54 is then able to detect the template within the image data provided by camera 52, to identify the face of the subject within the opening 18 of the template 10 and to recognize the relevant facial landmarks in the face of the subject in order to determine the relevant dimensions of the subject's face. Therefore, the processing unit 54 recognizes also any markings according to the present invention, like the markings 22, 24, 26 and 28 on template 10.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte and Lv by including a template for collecting data of a face of a subject that is taught by Znamenskiy, to make the invention that a system for collecting data of a face of a subject; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the quality and detailed measurements in the face of the subject as well as reducing time in image analysis such as a defined segmentation between the face and the background (Znamenskiy: Paragraph 30)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 15, Fonte, as modified by Lv discloses all the claims invention except wherein the feature includes a fixed element detected in a background area surrounding the head of the user.
Znamenskiy discloses the feature includes a fixed element (edges 49) detected in a background area surrounding the head of the user (Figs. 1-4 and Paragraph 115: “Similar to edges 46, edges 49 can be regarded as fiducial markers as well. Accordingly, a high contrast between the surface 16 and the background, in this embodiment in the opening 18, is beneficial for a reliable and exact detection of the edges 49 in an image of template 13, for example by a computer. Since the background is typically provided by a face of a subject in opening 18, such a contrast is advantageously generally given within this embodiment.”; Paragraph 107;)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte and Lv by including a template for collecting data of a face of a subject that is taught by Znamenskiy, to make the invention that a system for collecting data of a face of a subject; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the quality and detailed measurements in the face of the subject as well as reducing time in image analysis such as a defined segmentation between the face and the background (Znamenskiy: Paragraph 30)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 16, Fonte, as modified by Lv discloses all the claims invention except wherein the feature includes a plurality of features, including: at least one facial landmark defined by two facial features; and at least one element defined by at least two fixed key points detected in a background area surrounding the head of the user.
Znamenskiy discloses the feature includes a plurality of features, including: at least one facial landmark defined by two facial features; ; (Fig.7 and Paragraph 133: “the processing unit 54 may recognize facial landmarks like eyes 88, the nose 74 or the mouth 76 of the subject 70. Therefore, a first face detection step will be applied to the image of template 10 and face 72 of the subject 70.”) and at least one element defined by at least two fixed key points (Fig.4; plurality edges 49; Fig.1 :markings 22, 24, 26 and 28) detected in a background area surrounding the head of the user. (Figs. 1-4 and Paragraph 115: “Similar to edges 46, edges 49 can be regarded as fiducial markers as well. Accordingly, a high contrast between the surface 16 and the background, in this embodiment in the opening 18, is beneficial for a reliable and exact detection of the edges 49 in an image of template 13, for example by a computer. Since the background is typically provided by a face of a subject in opening 18, such a contrast is advantageously generally given within this embodiment.”; Paragraph 125: “The processing unit 54 is then able to detect the template within the image data provided by camera 52, to identify the face of the subject within the opening 18 of the template 10 and to recognize the relevant facial landmarks in the face of the subject in order to determine the relevant dimensions of the subject's face. Therefore, the processing unit 54 recognizes also any markings according to the present invention, like the markings 22, 24, 26 and 28 on template 10.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte and Lv by including a template for collecting data of a face of a subject that is taught by Znamenskiy, to make the invention that a system for collecting data of a face of a subject; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the quality and detailed measurements in the face of the subject as well as reducing time in image analysis such as a defined segmentation between the face and the background (Znamenskiy: Paragraph 30)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 23, Fonte, as modified by Lv discloses all the claims invention except wherein the feature includes a plurality of features, including: at least one facial landmark defined by at least two facial features; and element defined by at least two fixed key points detected in a background area surrounding the head of the user.
Znamenskiy discloses the feature includes a plurality of features, including: at least one facial landmark defined by at least two facial features; (Fig.7 and Paragraph 133: “the processing unit 54 may recognize facial landmarks like eyes 88, the nose 74 or the mouth 76 of the subject 70. Therefore, a first face detection step will be applied to the image of template 10 and face 72 of the subject 70.”) and element defined by at least two fixed key points (Fig.4; plurality edges 49; Fig.1 :markings 22, 24, 26 and 28) detected in a background area surrounding the head of the user(Figs. 1-4 and Paragraph 115: “Similar to edges 46, edges 49 can be regarded as fiducial markers as well. Accordingly, a high contrast between the surface 16 and the background, in this embodiment in the opening 18, is beneficial for a reliable and exact detection of the edges 49 in an image of template 13, for example by a computer. Since the background is typically provided by a face of a subject in opening 18, such a contrast is advantageously generally given within this embodiment.”; Paragraph 125: “The processing unit 54 is then able to detect the template within the image data provided by camera 52, to identify the face of the subject within the opening 18 of the template 10 and to recognize the relevant facial landmarks in the face of the subject in order to determine the relevant dimensions of the subject's face. Therefore, the processing unit 54 recognizes also any markings according to the present invention, like the markings 22, 24, 26 and 28 on template 10.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte and Lv by including a template for collecting data of a face of a subject that is taught by Znamenskiy, to make the invention that a system for collecting data of a face of a subject; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the quality and detailed measurements in the face of the subject as well as reducing time in image analysis such as a defined segmentation between the face and the background (Znamenskiy: Paragraph 30)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Claim(s) 9, 20 and 24-25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fonte et al (U.S. 20200285081 A1; Fonte), in view of Lv et al (U.S. 20130286161 A1; Lv), and in further view of Lucey et al (U.S. 20170173289 A1; Lucey).
Regarding claim 9, Fonte, as modified by Lv discloses all the claims invention except further comprising: repeatedly capturing the first image data and the second image data as the computing device is moved relative to the user to capture image data from a plurality of different positions and orientations of the computing device relative to the head of the user; correlating a plurality of changes in position and orientation of the computing device with a corresponding plurality of changes in position of the feature; determining a plurality of estimated scale values based on the correlating; and aggregating the plurality of estimated scale values to determine the scale value for sizing of the wearable device based on image data captured by the computing device.
Lucey discloses : repeatedly capturing the first image data and the second image data as the computing device is moved relative to the user to capture image data from a plurality of different positions and orientations of the computing device relative to the head of the user; (Paragraph 85: “Alternatively, instead of the user turning and tilting their head, the camera may be panned around the user's face, thus collecting profiles of the user's face from the same various angles. )
correlating a plurality of changes in position and orientation of the computing device with a corresponding plurality of changes in position of the feature; (Paragraph 91: “The motion data may be correlated to the video scan data … the video scan and motion data may be correlated based on their respective time information (e.g., timestamps, time delay information). The correlated motion data may then be used to correct or compensate the collected video scan data, effectively creating a video scan taken from a single point and/or angle of reference.”; Paragraph 100: “acceleration data and video data may be correlated with one another based further on properties or features of the collected data itself, instead of solely based on timestamps. For example, identified features of the user's face may be measured in the individual frames of the video data.”)
determining a plurality of estimated scale values based on the correlating; and aggregating the plurality of estimated scale values to determine the scale value for sizing of the wearable device based on image data captured by the computing device. (Paragraph 42; Paragraph 97: “The acceleration data may be correlated with the second video scan data, at this step or at a later step of processing (at 312), as with the optional correlation of the first video scan … The combination of the video scan data and acceleration data may be used to determine an estimated scale of the videoed subject”; Paragraph 121-122; Paragraph 137: “Knowing the acceleration of the camera from frame to frame (which itself involves correlating the acceleration data with the respective video frames), in combination with analyzing the relative size of the user as imaged from frame to frame, may provide sufficient information as to the user's actual distance from the camera during the second video scan. … Once the actual scale of the video frame (stated differently, the actual distance of the user in a given video frame) is known, a scaling factor of the user's face may be determined,”; Paragraph 48: “scaling the three-dimensional representation of the user's face … designing the custom-fit article based on the scaled three-dimensional representation of the user's face.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte and Lv by including Three-Dimensional Facial Scan Using End-User Device that is taught by Lucey, to make the invention that generation of a digital scan of a user's face such as for obtaining of a patient respiratory mask; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the tracking points and/or measuring of features of the user's face as well as enhancing the comfort, cost, efficacy, ease of use and manufacturability used in design and production of apparel for a user’s face. (Lucey: Paragraph 17 and Paragraph 126)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 20, Fonte, as modified by Lv discloses all the claims invention except wherein the instructions also cause the at least one processor to: repeatedly capture the first image data and the second image data as the computing device is moved relative to the user to capture image data from a plurality of different positions and orientations of the computing device relative to the head of the user; correlate a plurality of changes in position and orientation of the computing device with a corresponding plurality of changes in position of the feature; determine a plurality of estimated scale values based on the correlating; and aggregate the plurality of estimated scale values to determine the scale value for sizing of the wearable device based on image data captured by the computing device.
Lucey discloses repeatedly capture the first image data and the second image data as the computing device is moved relative to the user to capture image data from a plurality of different positions and orientations of the computing device relative to the head of the user; (Paragraph 85: “Alternatively, instead of the user turning and tilting their head, the camera may be panned around the user's face, thus collecting profiles of the user's face from the same various angles. ) correlate a plurality of changes in position and orientation of the computing device with a corresponding plurality of changes in position of the feature; (Paragraph 91: “The motion data may be correlated to the video scan data … the video scan and motion data may be correlated based on their respective time information (e.g., timestamps, time delay information). The correlated motion data may then be used to correct or compensate the collected video scan data, effectively creating a video scan taken from a single point and/or angle of reference.”; Paragraph 100: “acceleration data and video data may be correlated with one another based further on properties or features of the collected data itself, instead of solely based on timestamps. For example, identified features of the user's face may be measured in the individual frames of the video data.”) determine a plurality of estimated scale values based on the correlating; and aggregate the plurality of estimated scale values to determine the scale value for sizing of the wearable device based on image data captured by the computing device. (Paragraph 42; Paragraph 97: “The acceleration data may be correlated with the second video scan data, at this step or at a later step of processing (at 312), as with the optional correlation of the first video scan … The combination of the video scan data and acceleration data may be used to determine an estimated scale of the videoed subject”; Paragraph 121-122; Paragraph 137: “Knowing the acceleration of the camera from frame to frame (which itself involves correlating the acceleration data with the respective video frames), in combination with analyzing the relative size of the user as imaged from frame to frame, may provide sufficient information as to the user's actual distance from the camera during the second video scan. … Once the actual scale of the video frame (stated differently, the actual distance of the user in a given video frame) is known, a scaling factor of the user's face may be determined,”; Paragraph 48: “scaling the three-dimensional representation of the user's face … designing the custom-fit article based on the scaled three-dimensional representation of the user's face.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte and Lv by including Three-Dimensional Facial Scan Using End-User Device that is taught by Lucey, to make the invention that generation of a digital scan of a user's face such as for obtaining of a patient respiratory mask; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the tracking points and/or measuring of features of the user's face as well as enhancing the comfort, cost, efficacy, ease of use and manufacturability used in design and production of apparel for a user’s face. (Lucey: Paragraph 17 and Paragraph 126)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 24, Fonte, as modified by Lv discloses all the claims invention except further comprising determining a size of a portion of the head of the user from the scale value.
Lucey discloses further comprising determining a size of a portion of the head of the user from the scale value. (Paragraph 95: “the device may also collect scale estimation data from which a relative scale or size of the user's face may be estimated.” ; Paragraph 122: “Once the actual scale of the video frame (stated differently, the actual distance of the user in a given video frame) is known, a scaling factor of the user's face may be determined,”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte and Lv by including Three-Dimensional Facial Scan Using End-User Device that is taught by Lucey, to make the invention that generation of a digital scan of a user's face such as for obtaining of a patient respiratory mask; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the tracking points and/or measuring of features of the user's face as well as enhancing the comfort, cost, efficacy, ease of use and manufacturability used in design and production of apparel for a user’s face. (Lucey: Paragraph 17 and Paragraph 126)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 25, Fonte, as modified by Lv discloses all the claims invention except further comprising: determining a plurality of the scale values by capturing the first image data and the second image data a plurality of times at a plurality of positions of the computing device relative to the user; aggregating the plurality of the scale values to determine an aggregated scale value; and performing a virtual selection of the wearable device using the aggregated scale value.
Lucey discloses further comprising: determining a plurality of the scale values by capturing the first image data and the second image data a plurality of times at a plurality of positions of the computing device relative to the user; (Figs 4a-5a; Paragraph 91-93: “The motion data may be correlated to the video scan data … the video scan and motion data may be correlated based on their respective time information (e.g., timestamps, time delay information). … The motion data may include an instantaneous orientation measurement at the time of and corresponding to one of the collected video frames, and may further include one or more acceleration measurements during or between each of the collected video frames, in order to track tilting and movement of the camera during the first video scan.”)
aggregating the plurality of the scale values to determine an aggregated scale value; and performing a virtual selection of the wearable device using the aggregated scale value. Paragraph 42; Paragraph 97: “The acceleration data may be correlated with the second video scan data, at this step or at a later step of processing (at 312), as with the optional correlation of the first video scan … The combination of the video scan data and acceleration data may be used to determine an estimated scale of the videoed subject”; Paragraph 121-122; Paragraph 137: “Knowing the acceleration of the camera from frame to frame (which itself involves correlating the acceleration data with the respective video frames), in combination with analyzing the relative size of the user as imaged from frame to frame, may provide sufficient information as to the user's actual distance from the camera during the second video scan. … Once the actual scale of the video frame (stated differently, the actual distance of the user in a given video frame) is known, a scaling factor of the user's face may be determined,”; Paragraph 48: “scaling the three-dimensional representation of the user's face … designing the custom-fit article based on the scaled three-dimensional representation of the user's face.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Fonte and Lv by including Three-Dimensional Facial Scan Using End-User Device that is taught by Lucey, to make the invention that generation of a digital scan of a user's face such as for obtaining of a patient respiratory mask; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving the tracking points and/or measuring of features of the user's face as well as enhancing the comfort, cost, efficacy, ease of use and manufacturability used in design and production of apparel for a user’s face. (Lucey: Paragraph 17 and Paragraph 126)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Relevant Prior Art Directed to State of Art
Gousev et al (U.S. 20180173986 A1), “Light Source Modulation for Iris Size Adjustment”, teaches about methods, apparatuses, systems, and non-transitory computer readable medium are described for performing iris authentication for a user. It also teaches about a method comprises (a) capturing an initial image of an eye of the user, the eye including an inner circular boundary between a pupil region and an iris region, and an outer circular boundary between the iris region and a sclera region, (b) determining, from the initial image, a first size measurement indicative of a size of the inner circular boundary, (c) responsive to at least the first size measurement, modulating one or more visible light sources to output visible light toward the eye of the user, (d) capturing a subsequent image of the eye of the user during a period of pupillary response resulting from modulating the one or more visible light sources, (e) based on the subsequent image of the eye, obtaining an iris data record of the user, and (f) comparing the iris data record of the user to one or more registered iris data records to authenticate the user.
Ramachandran et al (U.S. 20140125700 A1), “Using a plurality of sensors for mapping and localization”, teaches about method of performing localization and mapping for a mobile device includes identifying geometric constraints associated with a current area at which the mobile device is located, obtaining at least one image of the current area captured by at least a first camera of the mobile device, obtaining data associated with the current area via at least one of a second camera of the mobile device or a sensor of the mobile device, and performing localization and mapping for the current area by applying the geometric constraints and the data associated with the current area to the at least one image.
Fung et al (U.S. 20150237479 A1), “Methods And Systems For Cross-Validating Sensor Data Acquired Using Sensors Of A Mobile Device”, teaches method involves receiving image data includes images representative of a motion of the mobile device and is determined using a first sensor of the plurality of sensors; receiving sensor data corresponds to the motion of the mobile device and is determined using a second sensor of the plurality of sensors; determining a first estimation of the motion of the mobile device based on the image data and the first timing information, and determining a second estimation of the motion of the mobile device based on the sensor data and the second timing information; determining whether the first estimation of the motion of the mobile device is within a threshold variance of the second estimation of the motion of the mobile device. The method then involves providing an output indicative of a validity of the first timing information and the second timing information based on whether the first estimation of the motion of the mobile device is within the threshold variance of the second estimation of the motion of the mobile device.
Kassner Moritz (W.O. 2020147948 A1),” Methods for generating calibration data for head-wearable devices and eye tracking system”, teaches about methods for generating data suitable for calibrating a head- wearable device such as a head-wearable spectacles device that may be used to detect one or more gaze-related parameters of a user, and an eye tracking system including a head- wearable device. It also teaches about method includes displaying a not translatory movement of an object on a display, using the scene camera to generate field images of a field of view of the user wearing the head-wearable device and instructed to look at the object and to mimic the not translatory movement, and using the first eye camera to generate first images of at least a portion of a first eye of the user while the user is expected to look at the object and mimic the not translatory movement, determining respective positions of the object in the field images, and using the determined positions of the object to determine for the first images respective ground truth values of at least one gaze- direction related parameter of the user.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Duy A Tran whose telephone number is (571)272-4887. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ONEAL R MISTRY can be reached at (313)-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUY TRAN/ Examiner, Art Unit 2674
/ONEAL R MISTRY/ Supervisory Patent Examiner, Art Unit 2674