Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objection
Claim 9 objected to because of the following informalities: misspelling. Claim 9 cites “audio queues” rather than “audio cues” as cited in the specification, paragraph 46. Appropriate correction is required.
Claim Rejection – 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 5, 6, 11, 12, 13, 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shroff (US20230032467A1) hereinafter referenced as Shroff, in view of the following: Peruch (US9912862B2), hereinafter referenced as Peruch and Hofer (US2004189849A1) hereinafter referenced as Hofer.
Regarding claim 1, Shroff teaches the following:
A method for obtaining information regarding a scene using a display free body wearable computing device, the method comprising: identifying a scope of interest in the scene as indicated by a user of the display free wearable computing a device;
“when executed by the one or more processors, cause the system to perform a method. The method includes receiving from a user of a smart glass [17] one or more wearable devices 100 - 1 (e.g., a smart glass) [26] … read user gesture to identify a region of interest in a scene viewed through a smart glass as disclosed above [21] … The system may or may not have a display for the user [22].” (Shroff, [17, 21, 22, 26])
Shroff teaches a method for obtaining information regarding a scene using wearable devices, the method comprising: identifying a region of interest in the scene as indicated by a user of a device.
Shroff fail to teach the following: obtaining an image capture plan based images; and actively guiding movement of the user based on the image capture plan to obtain a set of images depicting portions of the scene.
Peruch does and teaches the following:
obtaining an image capture plan based images based on the scope of interest;
“compute a guidance map in accordance with the one or more initial images to identify one or more next poses” (Peruch, [07])
Peruch teaches computing a guidance map based the one or more initial images identify next poses. The guidance map is inherently derived from initial images which necessarily reflect the portion of the scene being capture (reads on scope of interest).
Peruch BASE is analogous art with respect to Shroff because they are from the same field of endeavor, namely image capturing and processing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff to create a method for obtaining information regarding a scene using a wearable computing device, the method comprising: identifying a region of interest in the scene as indicated by a user of a device; computing a guidance map based the one or more initial images identify next poses. A person of ordinary skill in the art would do such in order to improve image scanning efficiency.
Shroff in view of Peruch do not teach active guiding movement of the user based on the image capture plan to obtain a set of images depicting portions of the scene.
But Hofer does. Hofer teaches the following:
and actively guiding movement of the user based on the image capture plan to obtain a set of images depicting portions of the scene.
“Disclosed is a panoramic sequence guide that can be used to guide a user in composing and capturing images to be used to create a composite panoramic image. In use, the guide tracks a viewed scene, determines the proper position of a composition guide relative to the viewed scene” (Hofer, [06])
Hofer teaches actively guiding user movement via a dynamic positioned composite guide, based on a planned sequence of scene portions to be captured, to obtain set images of the scene.
Hofer BASE is analogous art with respect to Shroff in view of Peruch because they are from the same field of endeavor, namely image capturing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch to create a method for obtaining information regarding a scene using smart glasses, the method comprising: identifying a region of interest in the scene as indicated by a user of a device; computing a guidance map based the one or more initial images identify next poses; actively guiding user movement via a dynamic positioned composite guide, based on a planned sequence of scene portions to be captured, to obtain set images of the scene. A person of ordinary skill in the art would do such in order to improve engagement, usability, and rewarding experiences.
Regarding claim 2, Shroff in view of Peruch and Hofer teach the method of claim 1 and additionally teaches the following. Shroff teaches:
wherein obtaining, using at least one camera of the display free body wearable computing device, a first image depicting: a first portion of the scene, and a gesture introduced into the scene by the user;
“the camera captures the scene, including some example hand gestures from the user. The frames including hand gestures” (Shroff, [25])
Shroff teaches capturing, using at least one camera of the wearable computing device, a first image depicting: the frame of the scene, and a gesture introduced into the scene by the user
and identifying, based on the gesture and the first portion of the scene, the scope of the interest in the scene.
and identify the region of interest based on a reading of the hand gesture from the user.” (Shroff, [25])
Shroff teaches identifying, based on the gesture and the frame of the scene, the region of the interest in the scene.
Claim 19 is rejected using the same rationale or bases as applied to claim 2.
Regarding claim 5, Shroff in view of Peruch and Hofer teaches the method of claim 2, and additionally teaches the following. Peruch teaches:
wherein obtaining the image capture plan comprises: identifying, based on the scope of the interest, a number of images usable to generate a three-dimensional model having a desired level of quality;
“compute a guidance map in accordance with the one or more initial images to identify one or more next poses; … and output the images captured by the camera to generate a three-dimensional model [07] ... To build a successful (or sufficiently useful) 3D model, aspects of embodiments of the present invention are directed to providing assistance or guidance to suggest a good path to move the camera during scanning [46]” (Peruch, [07, 46])
Peruch teaches wherein computing the guidance map comprises: identifying, based on the one or more initial images of the object to be scanned (reads upon the scope of the interest) a number of poses usable to generate sufficient three-dimensional model;
for each image of the number of images, identifying a location and field of view for obtaining the respective image based on the three-dimensional model having the desired level of quality.
“the system determines whether scanning should continue or whether there is sufficient data to generate a good 3D model. As discussed above, an incremental coverage map) ΔΩN+1 (p) is a function that associates each possible camera pose p from which a new view could be obtained … Note that a pose p=(x, O) represents both the location x and orientation O of the camera system at the time of capture.” (Peruch, [95])
Peruch teaches for each pose of the number of poses wherein each pose is defines the camera is placed and oriented at the moment of an image is captured (reads on for each image of the number of images), identifying a location and orientation for obtaining the respective pose based having sufficient data to generate a good 3D model.
Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch and Hofer with the feature of Peruch to wherein computing the guidance map comprises: identifying, based on the one or more initial images of the object to be scanned, which reads upon the scope of the interest, a number of poses usable to generate sufficient three-dimensional model; and for each pose of the number of poses, identifying a location and orientation for obtaining the respective pose based having sufficient data to generate a good 3D model. A person of ordinary skill in the art would do such in order to improve computationally and power efficient for image processing.
Regarding claim 6, Shroff in view of Peruch and Hofer teaches the method of claim 5, and additionally teaches the following. Peruch teaches:
wherein obtaining the image capture plan further comprises: obtaining, based on the number of images, the locations, and the fields of view, a motion plan that, when performed by the user, enables a set of images to be captured using the at least one camera that allow for the three-dimensional model having the desired level of quality to be generated.
“compute a guidance map in accordance with the one or more initial images to identify one or more next poses [07] … the system determines whether scanning should continue or whether there is sufficient data to generate a good 3D model. As discussed above, an incremental coverage map) ΔΩN+1 (p) is a function that associates each possible camera pose p from which a new view could be obtained, … Note that a pose p=(x, O) represents both the location x and orientation O [95] … The user 14 can freely move the scanning system 10 over a path that captures images of the scene from multiple different poses, thereby resulting in multiple views. The scanning system 10 can assist the user in moving the scanning system over a path that efficiently provides good results [49]” (Peruch, [ 07, 49, 95])
The system computes the next poses using prior images and a coverage map, which defines a location and orientation of the pose. These poses collectively determine where and how the images should be captured. The system guides the user along a path through these poses. Peruch teaches generating a sequence of camera pose (reads on image capture plan) based on analysis of the previously captured images via a coverage map (reads on the number of images), where each pose specifies a location and orientation (reads on field of view) and providing guidance to the user to follow a path through the poses. As the user moves along this path, images are captured from multiple viewpoints forming a set of suitable images for the 3D model.
Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch and Hofer with the feature of Peruch wherein computing the guidance map comprises: identifying, based on the one or more initial image, a location, and orientation for obtaining the respective pose, a plan, when performed by the user, enables a set of views to be captured using the at least one camera that allow for sufficient data to generate a good 3D model. A person of ordinary skill in the art would do such for more computationally and power efficient image processing.
Regarding claim 11, Shroff in view of Peruch and Hofer teaches the method of claim 1, and additionally teaches the following. Peruch teaches:
further comprising: providing computer-implemented services using the set of images
“The remote processor 18, typically having much higher processing power and storage capabilities than the scanning system 10, can quickly optimize and finalize the 3D scan, and can provide this service to multiple scanning systems.” (Peruch 51)
Peruch teaches providing a remote processor services using scans.
Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch and Hofer with the feature of Peruch to further comprise: providing computer-implemented services using the set of images. A person of ordinary skill in the art would do such in order to improve image processing efficiency.
Regarding claim 12, Shroff in view of Peruch and Hofer teaches the method of claim 11, and additionally teaches the following. Peruch teaches:
wherein providing the computer-implemented services comprises: generating a three-dimensional model using the set of images;
“The multiple views are combined to generate a 3D model [50] … generating the 3D model may be computationally expensive…transmit raw or partially processed data… to a remote (or “offline”) processor 18 [51]” (Peruch, [50, 51])
Peruch teaches providing the remote processor comprises: generating a three-dimensional model using the set of views;
obtaining an insight using the three-dimensional model;
“The resulting 3D model can then be used to estimate the available space or volume 12a of the cargo area of the truck” (Peruch, [50])
Peruch teaches obtaining an insight regarding the available space using the resulting three-dimensional model;
and using the insight to convey enhanced information regarding the scene to the user.
“The results can be presented to the users directly from the servers …After capturing the 3D scan of the boxes, as shown in FIG. 7A, resulting 3D model can be analyzed to count the number of boxes that are present. (Peruch, [51])
Peruch teaches using the insight to convey from the resulting 3D enhanced information regarding the scene to the user moreover the analyzing the number of boxes within the scene.
Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch and Hofer with the feature of Peruch for providing the remote processor comprises: generating a three-dimensional model using the set of views; obtaining an insight regarding the available space using the resulting three-dimensional model; using the insight to convey from the resulting 3D enhanced information regarding the scene to the user moreover the analyzing the number of boxes within the scene. A person of ordinary skill in the art would do such in order to improve image processing efficiency.
Regarding claim 13, Shroff in view of Peruch and Hofer teaches the method of claim 1 and additionally teaches the following. Shroff teaches:
wherein the display free body wearable computing device comprises: an integrated sensing and interaction component adapted to: be positioned symmetrically on two portions of a user’s head, be positioned between ears and eyes of the user, and capture a stereo image of at least a portion of a scene present in a field of view of the user;
“wearable devices with video camera accessories…In some embodiments, a system includes a smart glass [22] … a smart glass 300 to help the user adjust an FOV 338 of a first forward-looking camera 315 - 1 based on a region of interest for image capture, according to some embodiments. The scene 310 may be captured also by a second forward-looking camera 315 - 2 (hereinafter, first and second forward-looking cameras 315 - 1 and 315 - 2 will be referred to, hereinafter, as ‘cameras 315’ [34]”) (Shroff [28, 34])
PNG
media_image1.png
477
692
media_image1.png
Greyscale
FIG 3. (Shroff)
Shroff teaches the wearable computing device comprises: smart glasses adapted to: be positioned symmetrically on two portions of a user’s head, be positioned between ears and eyes of the user, and capture region of interest based on the field of view of the user and scene;
an integrated computing, powering, and securing portion;
“Smart glass 100 - 1 may include a frame 105 … Processor 112 is configured to identify a region of interest (ROI) of user 101” (Shroff, [27])
PNG
media_image2.png
405
306
media_image2.png
Greyscale
FIG. 1. (Shroff)
Shroff teaches of an integrated processor secured on the frames which provides the computational power for the computer system of the smart glasses;
and an adjustment member adapted to position the integrated sensing and interaction component with respect to the integrated computing, powering, and securing portion.
“Smart glass 100 - 1 may include a frame 105. A camera 115 (e.g., forward-looking) is mounted on frame 105. A sensing device 128 … Processor 112” (Shroff, [27])
PNG
media_image3.png
405
306
media_image3.png
Greyscale
FIG. 1. (Shroff)
Shroff teaches a frame adapted to position the multiple sensors with respect to the integrated processor which provides the computational power for the computer system of the smart glasses.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable Shroff in view of Peruch, Hofer, and Kumar (US20160091975A1), hereinafter referenced as Kumar.
Regarding claim 3, Shroff in view of Peruch and Hofer teaches the method of claim 2 but fail to teach wherein, in an instance of the obtaining of the first image where the gesture indicates interest of the user in an object present in the scene: the scope of the interest is limited to the object. Kumar teaches the following:
wherein, in an instance of the obtaining of the first image where the gesture indicates interest of the user in an object present in the scene: the scope of the interest is limited to the object.
“a region of interest may be selected by enclosing an object with a user's hand or hands. [05] … Once the region of interest is properly identified, the object within the region of interest may be cropped” (Kumar, [05, 18])
Kumar teaches an instance wherein, in an instance of the obtaining of the first image where the gesture indicates a region of interest is localized to the object in the region of interest: the region of interest is cropped based on the object.
Kumar BASE is analogous art with respect to Shroff in view of Peruch and Hofer because they are from the same field of endeavor, namely image capturing and processing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch and Hofer wherein an instance of the obtaining of the first image where the gesture indicates a region of interest is localized to the object in the region of interest: the region of interest is cropped based on the object. A person of ordinary skill in the art would do to improve user experience.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shroff in view of Peruch, Hofer, and Xu (US20130076913A1), hereinafter referenced as Xu.
Regarding claim 4, Shroff in view of Peruch and Hofer teaches the method of claim 2, and additionally teaches the following. Shroff teaches:
wherein, in an instance of the obtaining of the first image where the gesture indicates general interest: the scope of the interest is the field of view.
“the camera captures the scene … and identify the region of interest based on a reading of the hand gesture from the user … the hand gesture may be to increase, expand, pan, or shift an FOV of one or more of cameras 115 according to an ROI of user 101” (Shroff, [25, 30])
Shroff teaches wherein, in an instance of the obtaining of the first image where the gesture indicates general interest: the scope of the interest is the field of view.
Shroff in view of Peruch and Hofer fail to teach the scope of the interest is all of the scene.
But Xu does. Xu teaches the following:
the scope of the interest is all of the scene
“spectral sensor 604 is focused on a pre-defined area of interest such as all of scene 601 in field of view 614A and, upon a triggering event having occurred” (Xu, [62])
Xu teaches the area of interest is all of scene.
Xu BASE is analogous art with respect to Shroff in view of Peruch and Hofer because they are from the same field of endeavor, namely image processing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch and Hofer wherein, in an instance of the obtaining of the first image where the gesture indicates general interest: the area of interest is all of scene. A person of ordinary skill in the art would do such in order to improve computationally and power efficient for image processing.
Claim(s) 7, 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shroff in view of Peruch and Hofer and Said (US2017307719A1), hereinafter referenced as Said.
Regarding claim 7, Shroff in view of Peruch and Hofer teaches the method of claim 1, and additionally teaches the following. Peruch teaches:
wherein actively guiding movement of the user comprises: monitoring movement of the user;
“Aspects of embodiments of the invention are directed to monitoring the coverage and the measurement density of different patches of surfaces of the subject (e.g., an object or scene) being scanned.” (Peruch, [78])
Peruch teaches wherein monitoring of the user comprises: monitoring the coverage and the measurement density of different patches of surfaces of the subject (e.g., an object or scene) being scanned by the user.
an image capture plan
“compute a guidance map in accordance with the one or more initial images to identify one or more next poses” (Peruch, [07])
Peruch teaches a guidance map.
a motion plan
“The user 14 can freely move the scanning system 10 over a path that captures images of the scene from multiple different poses.” (Peruch, [49])
Peruch teaches a path plan.
Shroff in view of Peruch and Hofer do not teach the following: comparing the movement of the user to a motion plan indicated by the image capture plan; and in an instance of the comparing where the movement of the user diverges from the motion plan: providing, to the user, sensory feedback that indicates a change in the movement of the user, the change being adapted to conform the movement of the user to the motion plan.
Said does. Said teaches the following:
comparing the movement of the user to a motion plan indicated by the calculated route;
“The audio navigation system 25 may learn of a user deviation through the mobile device 100 encountering the beacon 42 that is not along the calculated route.” (Said, [66])
Said teaches comparing the mobile device, which is carried by the user as they walk, to beacons along calculated route;
in an instance of the comparing where the movement of the user diverges from a motion plan: providing, to the user, sensory feedback that indicates a change in the movement of the user, the change being adapted to conform the movement of the user to the motion plan.
“Audio may be generated by a text-to-speech component …When in a navigation mode, the system may give turn-by-turn directions while continuously updating the user on her present location and recalculating the route in the case the user deviates from the calculated route … At this point, the audio navigation system 25 may calculate a new route from the user's current location to the user to their desired destination.” (Said, [13-14, 66])
Said teaches of when the user deviates from the calculated route, the audio navigation system will provide turn-by-turn directions indicating the user deviated from the calculated route and the change is being recalculated based on user’s current location to the user to their desired destination.
Said teaches in an instance of the comparing where the user deviation from the calculated route, providing, to the user, audio navigation system feedback which includes turn-by-turn directions that indicates the user deviates from the calculated route, the change being recalculated to conform the user's current location to the user to their desired destination.
Said BASE is analogous art with respect to Shroff in view of Peruch and Hofer because they are from the same field of endeavor, namely image processing and guided movements. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch and Hofer wherein monitoring of the user comprises: monitoring the coverage and the measurement density of different patches of surfaces of the subject (e.g., an object or scene) being scanned by the user; comparing the mobile device, which is carried by the user as they walk, and beacons along calculated route, guidance map guidance map; and in an instance of the comparing where of the user deviation from the calculated route; providing, to the user, an audio navigation system feedback which includes turn-by-turn directions that indicates the user deviates from the path plan, the change being recalculated to conform the user's current location to the user to the path plan to the desired destination. A person of ordinary skill in the art would do such in order to improve user experience and useability.
Regarding claim 9, Shroff in view of the following: Peruch, Hofer, and Said teach the method of claim 7 teaches the method of claim 7, and additionally teaches the following. Said teaches:
wherein the sensory feedback comprises audio queues that convey the change in the movement to the user.
“Audio may be generated by a text-to-speech component that converts text generated by the system into audio announcements that are played back to the user [13] … the user deviates from the calculated route, the mobile device application 141 will immediately alert the user and the audio navigation system 25 will calculate a new route from the user's current location.” (Said, [13, 78])
Said teaches the audio navigation system comprises text-to-speech audio alerts that convey the user deviated from the route.
Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of the following: Peruch, Hofer, and Said with the feature of Said which teaches wherein audio navigation system comprises alerts that convey the user deviated from the route. A person of ordinary skill in the art would do such in order to improve user experience and useability.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable Shroff in view of Peruch and Hofer, Said, in view of Sommer (US11709068B2), hereinafter referenced as Sommer.
Regarding claim 8, Shroff in view of the following: Peruch, Hofer, and Said teach the method of claim 7, but does not teach the following: wherein the sensory feedback comprises spatial audio adapted to focus attention of the user on the scope of interest. But Sommer does. Sommer teaches:
wherein the sensory feedback comprises spatial audio adapted to focus attention of the user on the scope of interest.
“the system may recalculate a new path to the target location and begin directing the user on the new path using the spatial audio navigation methods.” (Sommer, 87)
Sommer teaches wherein the system comprises spatial audio navigation methods adapted to focus on the scope of interest of the user on the new path.
Sommer BASE is analogous art with respect to Shroff in view of Peruch and Hofer, Said because they are from the same field of endeavor, namely 3D image processing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Sommer to wherein the system comprises spatial audio navigation methods adapted to focus on the scope of interest of the user on the new path. A person of ordinary skill in the art would do such in order to improve user experience and useability.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shroff in view of Peruch, Hofer, Said, Li (US20170308772A1), hereinafter referenced as Li, and Stathacopoulos (CN109997094B), hereinafter referenced as Stathacopoulos.
Regarding claim 10, Shroff in view of the following: Peruch, Hofer, and Said teaches the method of claim 7, and additionally teaches the following. Hofer teaches:
wherein actively guiding movement of the user comprises: as the set of the images are obtained: comparing each image of the set of images to each other to determine the direction of capture;
“During tracking, relatively low resolution images of the viewed scene are captured in rapid succession and compared to determine the "movement" of objects in the scene relative to the camera orientation” (Hofer, [29])
Hofer teaches wherein tracking: as the set of images are captured in rapid succession: compared each image of the set of images to each other to determine the direction of capture.
Shroff teaches:
providing, to the user, second sensory feedback that indicates a change in the movement of the user’s head, the change being adapted to enable adjusting the user’s head.
“a feedback to the user based on a relative position between the region of interest and the field of view of the first forward-looking camera, the feedback intended to have the user adjust a head position relative to the object of interest [17] ... the system could provide feedback to the user indicating that ROI 238 is outside the FOV of one of forward-looking cameras 215 …This indication could be provided, for example, using audio 207 [32]” (Shroff, [17, 32])
Shroff teaches providing, to the user, audio feedback that indicates a change in the movement of the user’s head regarding the region of interest being outside the field of view, the change being adapted to enable adjusting the user head relative to the object of interest.
Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of the following: Peruch, Hofer, and Said with the features of Shroff and Hofer wherein tracking: as the set of images are captured in rapid succession: compared each image of the set of images to each other to determine the direction of capture; and providing, to the user, audio feedback that indicates a change in the movement of the user’s head regarding the region of interest being outside the field of view, the change being adapted to enable adjusting the user head relative to the object of interest. A person of ordinary skill in the art would do such in order to improve user experience.
Shroff in view of the following: Peruch, Hofer, and Said fail to teach the following: comparing each image of the set of images to a quality standard; in an instance of the comparing where an image of the set of images does not meet the quality standard: discarding the image of the images; providing, to the user, notification that indicates a change in user location, the change being adapted to enable obtaining of a replacement image for the image of the images.
But Li does. Li teaches
comparing each image of the set of images to a quality standard;
“comparing the qualities of a plurality of images when the number of images in the any same group is plural, and ranking the plurality of images according to the comparison result [22] … the quality of an image can be judged according to the characteristics (for example, definition) of the image so as to classify the image based on the quality of the image... the analyzing unit comprises a calculating unit for calculating the definition degree of the any image according to a preset definition degree calculating function [31-32]” (Li, [22, 31-32])
Li teaches comparing each image to definition degree.
in an instance of the comparing where an image of the set of images does not meet the quality standard: discarding the image of the images;
“the quality of an image can be judged according to the characteristics (for example, definition) of the image so as to classify the image based on the quality of the image, images of which the category meets quality requirements are recommended to be saved, and images of which the category does not meet the quality requirements are recommended to be deleted.” (Li, [32])
Li teaches in an instance of the comparing where an image of the set of images does not meet the quality requirements: deleting the image of the images;
Li BASE is analogous art with respect to Shroff in view of the following: Peruch, Hofer, and Said because they are from the same field of endeavor, namely image processing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of the following: Peruch, Hofer, and Said to wherein tracking: as the set of images are captured in rapid succession: comparing each image to definition degree; in an instance of the comparing where an image of the set of images does not meet the quality requirements: deleting the image of the images; providing, to the user, audio feedback that indicates a change in the movement of the user’s head regarding the region of interest being outside the field of view, the change being adapted to enable adjusting the user head relative to the object of interest. A person of ordinary skill in the art would do such in order to improve image processing efficiency.
However, Shroff in view of the following: Peruch, Hofer, Said, and Li fail to teach the following: providing, to the user, notification that indicates a change in user location, the change being adapted to enable obtaining of a replacement image for the image of the images.
But Stathacopoulos does. Stathacopoulos teaches:
providing, to the user, notification that indicates a change in user location, the change being adapted to enable obtaining of a replacement image for the image of the images
“generate another notification for the user, changing the user's device position to capture the next frame of the scene.” (Stathacopoulos, [24])
Stathacopoulos teaches another notification that indicates a change in user's device position, the change being adapted to enable capturing of the next frame of the scene.
Stathacopoulos BASE is analogous art with respect to Shroff in view of the following: Peruch, Hofer, Said and Li because they are from the same field of endeavor, namely image processing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of the following: Peruch, Hofer, Said and Li to wherein tracking: as the set of images are captured in rapid succession: compared each image of the set of images to compare each image to a comparison result; in an instance of the comparing where an image of the set of images does not meet the quality standard classifying them in second quality category: deleting the image of the images; providing, to the user, another notification audio feedback that indicates a change in user's device position, the change being adapted to enable capturing of the next frame of the scene. A person of ordinary skill in the art would do such in order to improve useability.
Claim(s) 14, 15, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shroff in view of the following: Peruch, Hofer, and Shin (US20230359271A1) hereinafter referenced as Shin.
Regarding claim 14, Shroff in view of Peruch and Hofer teaches the method of claim 13, and additionally teaches the following. Shroff teaches:
wherein the integrated sensing and interaction component comprises: a pair of cameras; speakers; a microphone array; and a touch pad.
“ a speaker 127 to communicate to user 101 voice or sound signals… In some embodiments, smart glass 100 - 1 may include multiple sensors 121 such as IMUs, gyroscopes, microphones, and capacitive sensors configured as touch interfaces for the user.[28] … of a first forward-looking camera 315 - 1 … a second forward-looking camera 315 - 2 (hereinafter, first and second forward-looking cameras 315 - 1 and 315 - 2 will be referred to, hereinafter, as ‘cameras 315’ [34]”) (Shroff, [28, 34])
Shroff teaches the smart glasses comprises: two cameras; a speaker; microphones; and a touch interface.
Shroff fails to teach speakers. But Shin does. Shin teaches:
Speakers
“the frame may include … interface devices (e.g., speakers, display, network adapter, etc.)” (Shin, [42])
Shin BASE is analogous art with respect to Shroff in view of Peruch and Hofer because they are from the same field of endeavor, namely image processing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch and Hofer wherein multiple sensors comprises: two cameras; speakers; microphones; and a touch interface. A person of ordinary skill in the art would do such in order to improve user experience.
Regarding claim 15, Shroff in view of Peruch, Hofer, and Shin teaches the method of claim 14, and additionally teaches the following. Shroff teaches:
wherein the integrated sensing and interaction component is adapted to: obtain the stereo image from the pair of cameras;
“of a first forward-looking camera 315 - 1 based on a region of interest for image capture, according to some embodiments. The scene 310 may be captured also by a second forward-looking camera 315 – 2” (Shroff, [34])
PNG
media_image1.png
477
692
media_image1.png
Greyscale
FIG 3. (Shroff)
Shroff teaches wherein smart glasses is adapted to: obtain the stereo image, via the multiple forward-looking cameras capturing the same scene which enables stereo imaging, from the pair of cameras;
and use at least the speakers to perform the action.
“a speaker 127 to communicate to user 101 voice or sound signals indicative of adjusting a gaze direction for improving the FOV of camera 115” (Shroff, 27)
Shroff teaches a speaker communicate to the user to adjust the gaze direction.
Peruch teaches:
at least partially process the stereo image to obtain an image processing result;
“In this process, a camera collects data from different views of an ordinary object, then aligns and combines the data to create a 3D model [03]… image can be obtained by different methods…stereo camera systems [55]” (Peruch, [03, 55])
Peruch teaches images are obtained via a stereo system, and at least partially process the stereo image to obtain an image processing result of a 3D model;
identify an action to be performed based, at least in part, on the image processing result and a derived result from a remote entity, the derived result being based, at least in part, on the stereo image and/or the image processing result;
“in accordance with the one or more initial images to identify one or more next poses; controlling, by the processor, the camera to capture one or more additional images from at least one of the one or more next poses … and outputting the images captured by the camera to generate a three-dimensional model. [18] … image can be obtained by different methods … stereo camera systems [55] … the process of finalizing the model in operation 310 includes transmitting some partially processed or raw data to a remote processor to generate the finalized 3D model. [113]” (Peruch, [18, 55, 103])
Peruch teaches identifying one or more next poses and controlling the camera to capture additional images. The next poses are determined based on prior captured images, which implies processing those images to guide the subsequent capture. The remote processor uses the transmitted image data to generate the finalized 3D model. Peruch teaches
Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch, Hofer, and Shin with the feature of Shroff and Peruch wherein the smart glasses is adapted to: obtain the stereo image, via the multiple forward-looking cameras capturing the same scene enabling stereo imaging, from the pair of cameras; at least partially process the stereo image to obtain an image processing result of a 3D model; identifying the next poses and controlling the camera to capture additional images based on prior image processing while also incorporating a derived result from a remote processor that uses transmitted image data to generate the final 3D model; and use at least a speaker to communicate to the user to adjust the gaze direction. A person of ordinary skill in the art would do such in order to improve image processing efficiency and user experience.
Regarding claim 17, Shroff in view of Peruch, Hofer, and Shin teaches the method of claim 14, and additionally teaches the following. Peruch teaches:
wherein the stereo image comprises a pair of images of the scene, each of the images being captured at different angles and/or positions with respect to the scene by the pair of cameras.
“stereoscopic depth camera system includes at least two cameras that are spaced apart from each other and rigidly mounted to a shared structure such as a rigid frame. The cameras are oriented in substantially the same direction … and have overlapping fields of view.” (Peruch, [63])
Peruch teaches wherein the stereo image comprises a pair of images of the scene captured by at least wo cameras spaced apart from each other with overlapping field of view.
Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch, Hofer, and Shin with the feature of Peruch wherein the stereo image comprises a pair of images of the scene captured by at least wo cameras spaced apart from each other with overlapping field of view. A person of ordinary skill in the art would do such in order to improve image processing efficiency.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shroff in view of the following: Peruch, Hofer, Shin, and Bleyer (US2021160441A1), hereinafter referenced as Bleyer
Regarding claim 16, Shroff in view of Peruch, Hofer, and Shin teaches the method of claim 14, and additionally teaches the following. Shroff teaches:
wherein the pair of cameras comprise lenses configured to: establish a camera line of sight that is converging to a line of sight of the user;
“in the eyepieces of the smart glass, two converging lines indicative of the gaze direction” (Shroff, [48])
Shroff teaches wherein the pair of cameras comprise lenses configured to: establish a camera line of sight that is converging to a line of sight of the user.
and establish a camera field of view that comprises the field of view of the user.
“Camera 515 faces forward towards the user FOV.” (Shroff [35])
Shroff teaches establishing a camera field of view that comprises the field of view of the user.
Shroff does not teach a camera line of sight that is parallel to a line of sight of the user. But Bleyer does. Bleyer teaches:
a camera line of sight that is parallel to a line of sight of the use
“the perspective appears as though those camera image(s) 315 were captured using cameras having parallel (or near parallel or at least corresponding to an alignment of human eyes) optical axes.” (Bleyer, [72])
Bleyer teaches a camera line of sight that is parallel to a line of sight of the user.
Bleyer BASE is analogous art with respect to Shroff in view of Peruch, Hofer, and Shin because they are from the same field of endeavor, namely image processing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch, Hofer, and Shin to wherein the pair of cameras comprise lenses configured to: establish a camera line of sight that is parallel to a line of sight of the user; establish a camera field of view that comprises the field of view of the user. A person of ordinary skill in the art would do such in order to improve image processing efficiency.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable Shroff in view of Peruch, Hofer, and Wexler (US2020053262A1), hereinafter referenced as Wexler.
Claim 18 is rejected using the same rationale or bases as applied to claim 1 and the mentioned structure. Additionally, claim 18 recites the following structure as taught by Wexler.
a non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations
“In one embodiment, a non-transitory computer-readable medium for use in a system employing a wearable image sensor pairable with a mobile communications device, the computer readable medium containing instructions. The instructions, when executed by at least one processor” (Wexler, [08])
Wexler BASE is analogous art with respect to Shroff in view of Peruch and Hofer because they are from the same field of endeavor, namely image processing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch and Hofer to create a non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations. A person of ordinary skill in the art would do such in order to improve image processing efficiency.
Claim(s) 20 is/are rejected under 35 U.S.C. 103 as being unpatentable Shroff in view of Peruch, Hofer, and Fisker (US8837026B2), hereinafter referenced as Fisker.
Claim 20 is rejected using the same rationale or bases as applied to claim 1 and the mentioned structure. Additionally, claim 20 recites the following structure as taught by Fisker.
A data processing system, comprising: a processor; and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations
“a data processing system as described above for producing a 3D computer model of a physical object, … a central processing unit, a memory… stored therein data representing sequences of instructions which when executed cause the method of the invention to be performed.” (Fisker, [47])
Fisker BASE is analogous art with respect to Shroff in view of Peruch and Hofer because they are from the same field of endeavor, namely image processing. Before the effective filling date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to modify Shroff in view of Peruch and Hofer to a data processing system, comprising: a processor; and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations. A person of ordinary skill in the art would do such in order to improve image processing efficiency.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUNE NGUYEN whose telephone number is (571)272-8919. The examiner can normally be reached M-TH 7:00AM - 5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona E Faulk can be reached at (571) 272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUNE NGOC NGUYEN/Examiner, Art Unit 2618
/DEVONA E FAULK/Supervisory Patent Examiner, Art Unit 2618
Prosecution Insights