DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-20 are pending under this Office action.
Claim Objections
Claims 2, 10, and 14 are objected to because of the following informalities: the term of “to satisfy a first brightness threshold” may be “to satisfy the first brightness threshold” because the term “a first brightness threshold” has been defined in the related independent claims 1, 9, and 13. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 9-11, and 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Atlas, etc. (US 11887263 B1) in view of Marzari, etc. (US 20220053142 A1), further in view of Takemoto (US 20140140579 A1) and Yamashita, etc. (US 20090167682 A1).
Regarding claim 1, Atlas teaches that a non-transitory computer readable medium comprising computer readable code executable by one or more processors (See Atlas: Fig. 14, and Col. 23 Lines 1-16, “FIG. 14 illustrates an example computer system 1400. In particular embodiments, one or more computer systems 1400 perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems 1400 provide functionality described or illustrated herein. In particular embodiments, software running on one or more computer systems 1400 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems 1400. Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate”) to:
receive an image data (See Atlas: Fig. 12, and Col. 18 Lines 59-64, “In particular embodiments, rendering adjustments may be made using depth data. In one embodiment, the depth data may be produced by a mobile device when photos are captured, for example. In another embodiment, the depth data may be captured, for example, in real-time or near real-time”. Note that the captured image input to the system may be mapped to the received image data) stream comprising a hand;
in accordance with a first brightness value for a first portion of the image data stream (See Atlas: Figs. 3A-H, and Col. 14 Lines 8-18, “In FIG. 3H, an emitted image 360 illustrates an example in which a real-world scene has been modified to increase the perceptibility of overlaid AR virtual content. In FIG. 3H, a brightness level of the background object area 362 has been reduced, for example, by using active dimming of the HMD 100 to reduce the transmission of scene light 116 through a lens in response to an electrical signal. Reducing the brightness of the background object area 362 increases the contrast with the content area 314, thereby increasing the perceptibility of the portion of the content area 314 that overlaps the background object area 362”; and Fig. 12, and Col. 15 Lines 8-24, “In particular embodiments, the computing device may determine that a visual enhancement is desired based on an assessment of whether the AR virtual content in the image, when displayed with the scene, would likely have poor perceptibility. The device may make such an assessment by comparing characteristics of the scene with characteristics with the image. For example, if the image having low additive contrast is an issue, and the underlying scene having high spatial variability is an issue, then the computing device may conclude that perceptibility may be an issue. Indeed, in some embodiments, one or more predetermined thresholds may be programmed into the computing device to selectively trigger when to perform adaptive rendering. For example, a look-ahead analysis of a particular scene may be performed to determine whether or not the potential perceptibility is favorable (e.g., independent of the image AR virtual content to be rendered)”. Note that the brightness level of the background object may be mapped to the brightness value of a first portion of the image) satisfies a first brightness threshold, perform a hands matting process in which attributes of the hand are identified using the image data stream; and
blend the image data stream with virtual content (See Atlas: Figs. 3A-H, and Col. 10 Lines 53-67 ~ Col. 11 Lines 1-14, “FIG. 3A illustrates an example scene 302 depicting a real-world object 304. The scene may be viewed by a user of a head-mounted display (HMD) 100 such as that shown in FIG. 1. An image of the real-world object 304 and scene light 116 may be directed to an eye 120 of a user for viewing. The real-world object 304 may be displayed on the display device 110 in a perceived image 162 with particular attributes that are perceptible to the eye 120. For example, the real-world object 304 may be displayed in particular colors, such as hues of green, with particular brightness. The HMD 100 may overlay AR virtual content 157 on the scene light 116. Thus, the real-world object 304 may appear to the eye 120 as a background object, particularly if the AR virtual content 157 overlaps the real-world object 304 on the perceived image 162, in which case the AR virtual content 157 may appear to be blended with the overlapped area of the real-world object 304. In some cases, depending on the colors, brightness, or other visible attributes of the real-world object 304 and the AR virtual content 157, the AR virtual content 157 and/or the real-world object 304 may be difficult for the eye 120 to see and identify in the perceived image 162. For example, if there is a lack of contrast between the AR virtual content 157 and the real-world object 304, the eye 120 may have difficulty distinguishing the AR virtual content 157 and the real-world object 304 from each other. As another example, if the color of the AR virtual content 157 is darker, then little or no light 155 may be projected by the projector 112, and the AR virtual object 157 is effectively transparent to the eye 120”) in accordance with the attributes of the hand.
However, Atlas fails to explicitly disclose that an image data stream comprising a hand; a first brightness value for a first portion of the image data stream satisfies a first brightness threshold, perform a hands matting process in which attributes of the hand are identified using the image data stream; and blend with virtual content in accordance with the attributes of the hand.
However, Marzari teaches that an image data stream comprising a hand (See Marzari: Figs. 1-2, and [0242], “In conjunction with touch screen 112, display controller 156, optical sensor(s) 164, optical sensor controller 158, contact/motion module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions to capture still images or video (including a video stream) and store them into memory 102, modify characteristics of a still image or video, or delete a still image or video from memory 102”; and [0181], “For example, in situations where the device or the component of the device is in contact with a surface of a user that is sensitive to touch (e.g., a finger, palm, or other part of a user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in physical characteristics of the device or the component of the device”); and
a first brightness value for a first portion of the image data stream satisfies a first brightness threshold (See Marzari: Figs. 6A-V, and [0379], “In some embodiments, the first predefined condition is met when an amount of light (e.g., amount of brightness (e.g., 20 lux, 5 lux)) in the field-of-view of the one or more cameras is below a second predetermined threshold (e.g., 20 lux), and the first control affordance is an affordance (e.g., 614b) (e.g., a selectable user interface object) for controlling a low-light capture mode. Providing a first control affordance that is an affordance for controlling a low-light capture mode when an amount of light in the field-of-view of the one or more cameras is below a second predetermined threshold provides a user with a quick and easy access to controlling the low-light capture mode when such control is likely to be needed and/or used. Reducing the number of inputs needed to perform an operation enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user to provide proper inputs and reducing user mistakes when operating/interacting with the device) which, additionally, reduces power usage and improves battery life of the device by enabling the user to use the device more quickly and efficiently. In some embodiments, the electronic device (e.g., 600) receives a user input corresponding to the selection of the affordance (e.g. 650d) for controlling a low-light capture mode, and, in response to receiving the user input, the electronic device can change the state (e.g., active (e.g., on), inactive (e.g., off)) of the low-light capture mode and/or display a user interface to change the state of the low-light capture mode”).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Atlas to have an image data stream comprising a hand; a first brightness value for a first portion of the image data stream satisfies a first brightness threshold as taught by Marzari in order to enhance operability of the device and makes user-device interface in efficient manner so as to reduce user mistakes when operating/interacting with the device, thus reducing power usage and quickly and efficiently improving battery life of the device by enabling the user to use the device (See Marzari: Fig. 7, and [0373], “The camera user interface also includes (706) a camera control region (e.g., 606), the camera control region including a plurality of control affordances (e.g., 620, 626) (e.g., a selectable user interface object) (e.g., proactive control affordance, a shutter affordance, a camera selection affordance, a plurality of camera mode affordances) for controlling a plurality of camera settings (e.g., flash, timer, filter effects, f-stop, aspect ratio, live photo, etc.) (e.g., changing a camera mode) (e.g., taking a photo) (e.g., activating a different camera (e.g., front-facing to rear-facing)). Providing a plurality of control affordances for controlling a plurality of camera settings in the camera control region enables a user to quickly and easily and change and/or manage the plurality of camera settings. Providing additional control options without cluttering the UI with additional displayed controls enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user to provide proper inputs and reducing user mistakes when operating/interacting with the device) which, additionally, reduces power usage and improves battery life of the device by enabling the user to use the device more quickly and efficiently”). Atlas teaches a method and system that may adjust the brightness of the background image or the virtual content characteristics to make the blended virtual content clear or more visible based on some kinds of thresholds to trigger the adaptive renderings of the composite virtual content; while Marzari teaches a system and method that may capture and manage the visual media contents of captured image data streams with hands in the images and using brightness threshold to manage the visual media. Therefore, it is obvious to one of ordinary skill in the art to modify Atlas by Marzari to have the captured images with hands in the images and use the brightness threshold to trigger the image blending process. The motivation to modify Atlas by Marzari is “Use of known technique to improve similar devices (methods, or products) in the same way”.
However, Atlas, modified by Marzari, fails to explicitly disclose that perform a hands matting process in which attributes of the hand are identified using the image data stream; and blend with virtual content in accordance with the attributes of the hand.
However, Takemoto teaches that perform a hands matting process in which attributes of the hand are identified using the image data stream (See Takemoto: Figs. 6-14, and [0051], “Further, the MR presentation system generates a finally corrected distance image 1420 by interpolating and extrapolating a partial area of the high reliability distance image 1410, if the area is defective compared to the original target object area included in the captured image 1401, within the region ranging to the contour line extracted from the captured image 1401. Through the above-mentioned processing flow, the MR presentation system corrects a distance measurement error of a moving target object using the captured image 1401 and the distance image 1405. An example method for generating the reliability image 1305, the high reliability distance image 1410, and the corrected distance image 1420 is described in detail below”; and [0109], “Next, in step S311, the virtual image generating unit 110 generates a virtual object image using three-dimensional model information of the virtual object and the corrected distance image stored in the storage unit 109. According to the example using the corrected distance image 1420 illustrated in FIG. 14, first, the virtual image generating unit 110 renders the three-dimensional model information of the virtual object and generates color information about the virtual object image together with the Z buffer value. In the present exemplary embodiment, it is presumed that the virtual object image is rendered in such a way as to have a resolution comparable to that of the captured image 1401. However, the resolution of the captured image is not necessarily equal to the resolution of the virtual object image. It is useful to apply scaling transformation to the captured image according to the resolution of the virtual object image”. Note that the various hand image generations may be mapped to the hands matting process in which the hands attributes, e.g., the contours of the hands or nails, are identified).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Atlas to have perform a hands matting process in which attributes of the hand are identified using the image data stream as taught by Takemoto in order to enable the user to feel as if a virtual object that does not exist in real space, such as computer aided design (CAD) model, being actually present there (See Takemoto: Fig. 1, and [0036], “The MR presentation system is configured to present a composite image, which can be obtained by combining a real space image with a virtual space image (e.g., a computer graphics image), to a user (i.e., an MR experiencing person). Presenting such a composite image enables a user to feel as if a virtual object that does not exist in the real space, such as a computer aided design (CAD) model, were actually present there. The MR technique is, for example, discussed in detail in H. Tamura, H. Yamamoto and A. Katayama: "Mixed reality: Future dreams seen at the border between real and virtual worlds," Computer Graphics and Applications, vol.21, no.6, pp.64-70, 2001”). Atlas teaches a method and system that may adjust the brightness of the background image or the virtual content characteristics to make the blended virtual content clear or more visible based on some kinds of thresholds to trigger the adaptive renderings of the composite virtual content; while Takemoto teaches a system and method that may manipulate the characteristics of hands in the image in order to mix the hand images to the virtual content to generate high reliability images. Therefore, it is obvious to one of ordinary skill in the art to modify Atlas by Takemoto to have hands mattering process applied to the hand images with hands attributes identified. The motivation to modify Atlas by Takemoto is “Use of known technique to improve similar devices (methods, or products) in the same way”.
However, Atlas, modified by Marzari and Takemoto, fails to explicitly disclose that blend with virtual content in accordance with the attributes of the hand.
However, Yamashita teaches that blend with virtual content in accordance with the attributes of the hand (See Yamashita: Figs. 27A-H, and [0162], “FIG. 27C shows a screen example in the case where the body image generated by the body shape extraction section 600 is processed such that the contour of the body image is made non-transparent and the inside of the body image is made semi-transparent, and then the processed body image is drawn over the display information generated by the display information generation section 700. Based on the above-described image composition, it is possible to perform an intuitive input operation and also it is easy to confirm the display information even during the operation”. Note that the hand image is blended into the virtual content of the GUI component, and the hand features, e.g., hand contours and fingers, are composited and displayed in the composited (blended) images as shown in Figs. 27A-H).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Atlas to have blend with virtual content in accordance with the attributes of the hand as taught by Yamashita in order to detect hand shape and gesture accurately even if the user's hand protrudes outside the operation surface (See Yamashita: Fig. 1, and [0008], “The present invention is directed to solving the above problems. That is, an object of the present invention is to provide an input device capable of, even when a user's hand goes beyond an operation surface, performing input by accurately sensing a hand shape and a gesture, and its method”). Atlas teaches a method and system that may adjust the brightness of the background image or the virtual content characteristics to make the blended virtual content clear or more visible based on some kinds of thresholds to trigger the adaptive renderings of the composite virtual content; while Yamashita teaches a system and method that may generate and display the blended images by compositing the hands images with various hands features processed (matting) and the GUI components. Therefore, it is obvious to one of ordinary skill in the art to modify Atlas by Yamashita to have blending the hands and the virtual content according to the hands attributes. The motivation to modify Atlas by Yamashita is “Use of known technique to improve similar devices (methods, or products) in the same way”.
Regarding claim 2, Atlas, Marzari, Takemoto, and Yamashita teach all the features with respect to claim 1 as outlined above. Further, Atlas and Marzari teach that the non-transitory computer readable medium of claim 1, further comprising computer readable code to:
in accordance with a determination that that a second brightness value for a second portion of the image data stream fails to satisfy a first brightness threshold (See Atlas: Fig. 12, and Col. 15 Lines 8-24, “In particular embodiments, the computing device may determine that a visual enhancement is desired based on an assessment of whether the AR virtual content in the image, when displayed with the scene, would likely have poor perceptibility. The device may make such an assessment by comparing characteristics of the scene with characteristics with the image. For example, if the image having low additive contrast is an issue, and the underlying scene having high spatial variability is an issue, then the computing device may conclude that perceptibility may be an issue. Indeed, in some embodiments, one or more predetermined thresholds may be programmed into the computing device to selectively trigger when to perform adaptive rendering. For example, a look-ahead analysis of a particular scene may be performed to determine whether or not the potential perceptibility is favorable (e.g., independent of the image AR virtual content to be rendered)”. Note that the brightness level of the background object may be mapped to the brightness value of a first portion of the image), disable the hands matting process (See Marzari: Fig. 1-2, and [0042], “In accordance with some embodiments, a method is described. The method is performed at an electronic device having a display device and one or more cameras. The method comprises: receiving a request to display a camera user interface; and in response to receiving the request to display the camera user interface, displaying, via the display device, a camera user interface that includes: displaying, via the display device, a representation of a field-of-view of the one or more cameras; and in accordance with a determination that low-light conditions have been met, wherein the low-light conditions include a condition that is met when ambient light in the field-of-view of the one or more cameras is below a respective threshold, displaying, concurrently with the representation of the field-of-view of the one or more cameras, a control for adjusting a capture duration for capturing media in response to a request to capture media; and in accordance with a determination that the low-light conditions have not been met, forgoing display of the control for adjusting the capture duration”. Note that low-light condition not met may be mapped to fail to satisfy the first brightness threshold, and forgoing the adjustment control may be mapped to disable the hand matting process: Atlas teaches that the threshold is used to trigger whether adaptive rendering is employed or not, and Marzari teaches that if the low-light condition not met, adjustment of the image setting is disabled, thus, combining Atlas and Marzari will arrive at the scenario that the low-light condition not met, the adaptive rendering will be disabled, and the hand matting will be disabled).
Regarding claim 3, Atlas, Marzari, Takemoto, and Yamashita teach all the features with respect to claim 1 as outlined above. Further, Marzari and Yamashita teach that the non-transitory computer readable medium of claim 1, wherein the hands matting process is performed using a first hands matting model in accordance with the first brightness value satisfying the first brightness threshold (See Yamashita: Figs. 12-18, and [0116], “A second extraction method is a brightness threshold method. In the brightness threshold method, the body area is extracted by comparing the brightness value of each pixel of the image data outputted from the body shape input section 100 to a predetermined threshold”; and [0119], “When the extraction of the body area is completed, next, the body shape extraction section 600 extracts, based on the body area 607 extracted in step S604, a contour 609 of the extracted body area 607 as shown in FIG. 15 (step S606). This contour extraction process is performed by extracting, from the pixels included in the body area 607, pixels adjacent to the pixels of an area other than the body area 607. More specifically, from all of the pixels included in the body area 607, pixels each having four adjacent pixels above and below, to the left and right (or eight adjacent pixels further including the diagonally upper-right, diagonally upper-left, diagonally lower-right and diagonally lower-left pixels of the attention pixel) which include a pixel of an area other than the body area 607, are extracted. Note that a smoothing process may be performed on the extracted contour 609 as necessary. It is possible, by performing the smoothing process, to eliminate aliasing occurring in the contour 609”. Note that the hand contours may be mapped to the first hands matting model, and Fig. 16 with hands correction or transformation may be mapped to the second hands matting model), and further comprising computer readable code to:
in accordance with a determination that a second brightness value for a second portion of the image data stream fails to satisfy a first brightness threshold, perform the hands matting process on the image data stream using a second hands matting model (See Marzari: Figs. 5A-H, and [0322], “For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this example, a contact with a characteristic intensity that does not exceed the first threshold results in a first operation, a contact with a characteristic intensity that exceeds the first intensity threshold and does not exceed the second intensity threshold results in a second operation, and a contact with a characteristic intensity that exceeds the second threshold results in a third operation. In some embodiments, a comparison between the characteristic intensity and one or more thresholds is used to determine whether or not to perform one or more operations (e.g., whether to perform a respective operation or forgo performing the respective operation), rather than being used to determine whether to perform a first operation or a second operation”. Note that it is logical to have three different operations if there are two thresholds: first operation if the intensity below the first threshold; second operation if the intensity is between the first and second thresholds; and third operation if the intensity is greater than the second threshold, assume the first threshold is less than the second threshold. In the same way, if there are four thresholds, then, it may have five different operations, and so on).
Regarding claim 9, Atlas, Marzari, Takemoto, and Yamashita teach all the features with respect to claim 1 as outlined above. Further, Atlas, Marzari, Takemoto, and Yamashita teach that a system (See Atlas: Fig. 14, and Col. 23 Lines 1-16, “FIG. 14 illustrates an example computer system 1400. In particular embodiments, one or more computer systems 1400 perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems 1400 provide functionality described or illustrated herein. In particular embodiments, software running on one or more computer systems 1400 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems 1400. Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate”) comprising:
one or more processors (See Atlas: Fig. 14, and Col. 23 Lines 45-53, “In particular embodiments, computer system 1400 includes a processor 1402, memory 1404, storage 1406, an input/output (I/O) interface 1408, a communication interface 1410, and a bus 1412. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement”); and
one or more computer readable media comprising computer readable code executable by the one or more processors (See Atlas: Fig. 14, and Col. 23 Lines 45-53, “In particular embodiments, computer system 1400 includes a processor 1402, memory 1404, storage 1406, an input/output (I/O) interface 1408, a communication interface 1410, and a bus 1412. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement”) to:
receive an image data (See Atlas: Fig. 12, and Col. 18 Lines 59-64, “In particular embodiments, rendering adjustments may be made using depth data. In one embodiment, the depth data may be produced by a mobile device when photos are captured, for example. In another embodiment, the depth data may be captured, for example, in real-time or near real-time”. Note that the captured image input to the system may be mapped to the received image data) stream comprising a hand (See Marzari: Figs. 1-2, and [0242], “In conjunction with touch screen 112, display controller 156, optical sensor(s) 164, optical sensor controller 158, contact/motion module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions to capture still images or video (including a video stream) and store them into memory 102, modify characteristics of a still image or video, or delete a still image or video from memory 102”; and [0181], “For example, in situations where the device or the component of the device is in contact with a surface of a user that is sensitive to touch (e.g., a finger, palm, or other part of a user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in physical characteristics of the device or the component of the device”);
in accordance with a first brightness value for a first portion of the image data stream (See Atlas: Figs. 3A-H, and Col. 14 Lines 8-18, “In FIG. 3H, an emitted image 360 illustrates an example in which a real-world scene has been modified to increase the perceptibility of overlaid AR virtual content. In FIG. 3H, a brightness level of the background object area 362 has been reduced, for example, by using active dimming of the HMD 100 to reduce the transmission of scene light 116 through a lens in response to an electrical signal. Reducing the brightness of the background object area 362 increases the contrast with the content area 314, thereby increasing the perceptibility of the portion of the content area 314 that overlaps the background object area 362”; and Fig. 12, and Col. 15 Lines 8-24, “In particular embodiments, the computing device may determine that a visual enhancement is desired based on an assessment of whether the AR virtual content in the image, when displayed with the scene, would likely have poor perceptibility. The device may make such an assessment by comparing characteristics of the scene with characteristics with the image. For example, if the image having low additive contrast is an issue, and the underlying scene having high spatial variability is an issue, then the computing device may conclude that perceptibility may be an issue. Indeed, in some embodiments, one or more predetermined thresholds may be programmed into the computing device to selectively trigger when to perform adaptive rendering. For example, a look-ahead analysis of a particular scene may be performed to determine whether or not the potential perceptibility is favorable (e.g., independent of the image AR virtual content to be rendered)”. Note that the brightness level of the background object may be mapped to the brightness value of a first portion of the image) satisfies a first brightness threshold (See Marzari: Figs. 6A-V, and [0379], “In some embodiments, the first predefined condition is met when an amount of light (e.g., amount of brightness (e.g., 20 lux, 5 lux)) in the field-of-view of the one or more cameras is below a second predetermined threshold (e.g., 20 lux), and the first control affordance is an affordance (e.g., 614b) (e.g., a selectable user interface object) for controlling a low-light capture mode. Providing a first control affordance that is an affordance for controlling a low-light capture mode when an amount of light in the field-of-view of the one or more cameras is below a second predetermined threshold provides a user with a quick and easy access to controlling the low-light capture mode when such control is likely to be needed and/or used. Reducing the number of inputs needed to perform an operation enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user to provide proper inputs and reducing user mistakes when operating/interacting with the device) which, additionally, reduces power usage and improves battery life of the device by enabling the user to use the device more quickly and efficiently. In some embodiments, the electronic device (e.g., 600) receives a user input corresponding to the selection of the affordance (e.g. 650d) for controlling a low-light capture mode, and, in response to receiving the user input, the electronic device can change the state (e.g., active (e.g., on), inactive (e.g., off)) of the low-light capture mode and/or display a user interface to change the state of the low-light capture mode”),
perform a hands matting process in which attributes of the hand are identified using the image data stream (See Takemoto: Figs. 6-14, and [0051], “Further, the MR presentation system generates a finally corrected distance image 1420 by interpolating and extrapolating a partial area of the high reliability distance image 1410, if the area is defective compared to the original target object area included in the captured image 1401, within the region ranging to the contour line extracted from the captured image 1401. Through the above-mentioned processing flow, the MR presentation system corrects a distance measurement error of a moving target object using the captured image 1401 and the distance image 1405. An example method for generating the reliability image 1305, the high reliability distance image 1410, and the corrected distance image 1420 is described in detail below”; and [0109], “Next, in step S311, the virtual image generating unit 110 generates a virtual object image using three-dimensional model information of the virtual object and the corrected distance image stored in the storage unit 109. According to the example using the corrected distance image 1420 illustrated in FIG. 14, first, the virtual image generating unit 110 renders the three-dimensional model information of the virtual object and generates color information about the virtual object image together with the Z buffer value. In the present exemplary embodiment, it is presumed that the virtual object image is rendered in such a way as to have a resolution comparable to that of the captured image 1401. However, the resolution of the captured image is not necessarily equal to the resolution of the virtual object image. It is useful to apply scaling transformation to the captured image according to the resolution of the virtual object image”. Note that the various hand image generations may be mapped to the hands matting process in which the hands attributes, e.g., the contours of the hands or nails, are identified); and
blend the image data stream with virtual content (See Atlas: Figs. 3A-H, and Col. 10 Lines 53-67 ~ Col. 11 Lines 1-14, “FIG. 3A illustrates an example scene 302 depicting a real-world object 304. The scene may be viewed by a user of a head-mounted display (HMD) 100 such as that shown in FIG. 1. An image of the real-world object 304 and scene light 116 may be directed to an eye 120 of a user for viewing. The real-world object 304 may be displayed on the display device 110 in a perceived image 162 with particular attributes that are perceptible to the eye 120. For example, the real-world object 304 may be displayed in particular colors, such as hues of green, with particular brightness. The HMD 100 may overlay AR virtual content 157 on the scene light 116. Thus, the real-world object 304 may appear to the eye 120 as a background object, particularly if the AR virtual content 157 overlaps the real-world object 304 on the perceived image 162, in which case the AR virtual content 157 may appear to be blended with the overlapped area of the real-world object 304. In some cases, depending on the colors, brightness, or other visible attributes of the real-world object 304 and the AR virtual content 157, the AR virtual content 157 and/or the real-world object 304 may be difficult for the eye 120 to see and identify in the perceived image 162. For example, if there is a lack of contrast between the AR virtual content 157 and the real-world object 304, the eye 120 may have difficulty distinguishing the AR virtual content 157 and the real-world object 304 from each other. As another example, if the color of the AR virtual content 157 is darker, then little or no light 155 may be projected by the projector 112, and the AR virtual object 157 is effectively transparent to the eye 120”) in accordance with the attributes of the hand (See Yamashita: Figs. 27A-H, and [0162], “FIG. 27C shows a screen example in the case where the body image generated by the body shape extraction section 600 is processed such that the contour of the body image is made non-transparent and the inside of the body image is made semi-transparent, and then the processed body image is drawn over the display information generated by the display information generation section 700. Based on the above-described image composition, it is possible to perform an intuitive input operation and also it is easy to confirm the display information even during the operation”. Note that the hand image is blended into the virtual content of the GUI component, and the hand features, e.g., hand contours and fingers, are composited and displayed in the composited (blended) images as shown in Figs. 27A-H).
Regarding claim 10, Atlas, Marzari, Takemoto, and Yamashita teach all the features with respect to claim 9 as outlined above. Further, Atlas and Marzari teach that the system of claim 9, further comprising computer readable code to:
in accordance with a determination that that a second brightness value for a second portion of the image data stream fails to satisfy a first brightness threshold (See Atlas: Fig. 12, and Col. 15 Lines 8-24, “In particular embodiments, the computing device may determine that a visual enhancement is desired based on an assessment of whether the AR virtual content in the image, when displayed with the scene, would likely have poor perceptibility. The device may make such an assessment by comparing characteristics of the scene with characteristics with the image. For example, if the image having low additive contrast is an issue, and the underlying scene having high spatial variability is an issue, then the computing device may conclude that perceptibility may be an issue. Indeed, in some embodiments, one or more predetermined thresholds may be programmed into the computing device to selectively trigger when to perform adaptive rendering. For example, a look-ahead analysis of a particular scene may be performed to determine whether or not the potential perceptibility is favorable (e.g., independent of the image AR virtual content to be rendered)”. Note that the brightness level of the background object may be mapped to the brightness value of a first portion of the image), disable the hands matting process (See Marzari: Fig. 1-2, and [0042], “In accordance with some embodiments, a method is described. The method is performed at an electronic device having a display device and one or more cameras. The method comprises: receiving a request to display a camera user interface; and in response to receiving the request to display the camera user interface, displaying, via the display device, a camera user interface that includes: displaying, via the display device, a representation of a field-of-view of the one or more cameras; and in accordance with a determination that low-light conditions have been met, wherein the low-light conditions include a condition that is met when ambient light in the field-of-view of the one or more cameras is below a respective threshold, displaying, concurrently with the representation of the field-of-view of the one or more cameras, a control for adjusting a capture duration for capturing media in response to a request to capture media; and in accordance with a determination that the low-light conditions have not been met, forgoing display of the control for adjusting the capture duration”. Note that low-light condition not met may be mapped to fail to satisfy the first brightness threshold, and forgoing the adjustment control may be mapped to disable the hand matting process: Atlas teaches that the threshold is used to trigger whether adaptive rendering is employed or not, and Marzari teaches that if the low-light condition not met, adjustment of the image setting is disabled, thus, combining Atlas and Marzari will arrive at the scenario that the low-light condition not met, the adaptive rendering will be disabled, and the hand matting will be disabled).
Regarding claim 11, Atlas, Marzari, Takemoto, and Yamashita teach all the features with respect to claim 9 as outlined above. Further, Marzari and Yamashita teach that the system of claim 9, wherein the hands matting process is performed using a first hands matting model in accordance with the first brightness value satisfying the first brightness threshold (See Yamashita: Figs. 12-18, and [0116], “A second extraction method is a brightness threshold method. In the brightness threshold method, the body area is extracted by comparing the brightness value of each pixel of the image data outputted from the body shape input section 100 to a predetermined threshold”; and [0119], “When the extraction of the body area is completed, next, the body shape extraction section 600 extracts, based on the body area 607 extracted in step S604, a contour 609 of the extracted body area 607 as shown in FIG. 15 (step S606). This contour extraction process is performed by extracting, from the pixels included in the body area 607, pixels adjacent to the pixels of an area other than the body area 607. More specifically, from all of the pixels included in the body area 607, pixels each having four adjacent pixels above and below, to the left and right (or eight adjacent pixels further including the diagonally upper-right, diagonally upper-left, diagonally lower-right and diagonally lower-left pixels of the attention pixel) which include a pixel of an area other than the body area 607, are extracted. Note that a smoothing process may be performed on the extracted contour 609 as necessary. It is possible, by performing the smoothing process, to eliminate aliasing occurring in the contour 609”. Note that the hand contours may be mapped to the first hands matting model, and Fig. 16 with hands correction or transformation may be mapped to the second hands matting model), and further comprising computer readable code to:
in accordance with a determination that a second brightness value for a second portion of the image data stream fails to satisfy a first brightness threshold, perform the hands matting process on the image data stream using a second hands matting model (See Marzari: Figs. 5A-H, and [0322], “For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this example, a contact with a characteristic intensity that does not exceed the first threshold results in a first operation, a contact with a characteristic intensity that exceeds the first intensity threshold and does not exceed the second intensity threshold results in a second operation, and a contact with a characteristic intensity that exceeds the second threshold results in a third operation. In some embodiments, a comparison between the characteristic intensity and one or more thresholds is used to determine whether or not to perform one or more operations (e.g., whether to perform a respective operation or forgo performing the respective operation), rather than being used to determine whether to perform a first operation or a second operation”. Note that it is logical to have three different operations if there are two thresholds: first operation if the intensity below the first threshold; second operation if the intensity is between the first and second thresholds; and third operation if the intensity is greater than the second threshold, assume the first threshold is less than the second threshold. In the same way, if there are four thresholds, then, it may have five different operations, and so on).
Regarding claim 13, Atlas, Marzari, Takemoto, and Yamashita teach all the features with respect to claim 1 as outlined above. Further, Atlas, Marzari, Takemoto, and Yamashita teach that a method (See Atlas: Fig. 14, and Col. 23 Lines 1-16, “FIG. 14 illustrates an example computer system 1400. In particular embodiments, one or more computer systems 1400 perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems 1400 provide functionality described or illustrated herein. In particular embodiments, software running on one or more computer systems 1400 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems 1400. Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate”) comprising:
receiving an image data (See Atlas: Fig. 12, and Col. 18 Lines 59-64, “In particular embodiments, rendering adjustments may be made using depth data. In one embodiment, the depth data may be produced by a mobile device when photos are captured, for example. In another embodiment, the depth data may be captured, for example, in real-time or near real-time”. Note that the captured image input to the system may be mapped to the received image data) stream comprising a hand (See Marzari: Figs. 1-2, and [0242], “In conjunction with touch screen 112, display controller 156, optical sensor(s) 164, optical sensor controller 158, contact/motion module 130, graphics module 132, and image management module 144, camera module 143 includes executable instructions to capture still images or video (including a video stream) and store them into memory 102, modify characteristics of a still image or video, or delete a still image or video from memory 102”; and [0181], “For example, in situations where the device or the component of the device is in contact with a surface of a user that is sensitive to touch (e.g., a finger, palm, or other part of a user's hand), the tactile output generated by the physical displacement will be interpreted by the user as a tactile sensation corresponding to a perceived change in physical characteristics of the device or the component of the device”);
in accordance with a first brightness value for a first portion of the image data stream (See Atlas: Figs. 3A-H, and Col. 14 Lines 8-18, “In FIG. 3H, an emitted image 360 illustrates an example in which a real-world scene has been modified to increase the perceptibility of overlaid AR virtual content. In FIG. 3H, a brightness level of the background object area 362 has been reduced, for example, by using active dimming of the HMD 100 to reduce the transmission of scene light 116 through a lens in response to an electrical signal. Reducing the brightness of the background object area 362 increases the contrast with the content area 314, thereby increasing the perceptibility of the portion of the content area 314 that overlaps the background object area 362”; and Fig. 12, and Col. 15 Lines 8-24, “In particular embodiments, the computing device may determine that a visual enhancement is desired based on an assessment of whether the AR virtual content in the image, when displayed with the scene, would likely have poor perceptibility. The device may make such an assessment by comparing characteristics of the scene with characteristics with the image. For example, if the image having low additive contrast is an issue, and the underlying scene having high spatial variability is an issue, then the computing device may conclude that perceptibility may be an issue. Indeed, in some embodiments, one or more predetermined thresholds may be programmed into the computing device to selectively trigger when to perform adaptive rendering. For example, a look-ahead analysis of a particular scene may be performed to determine whether or not the potential perceptibility is favorable (e.g., independent of the image AR virtual content to be rendered)”. Note that the brightness level of the background object may be mapped to the brightness value of a first portion of the image) satisfies a first brightness threshold (See Marzari: Figs. 6A-V, and [0379], “In some embodiments, the first predefined condition is met when an amount of light (e.g., amount of brightness (e.g., 20 lux, 5 lux)) in the field-of-view of the one or more cameras is below a second predetermined threshold (e.g., 20 lux), and the first control affordance is an affordance (e.g., 614b) (e.g., a selectable user interface object) for controlling a low-light capture mode. Providing a first control affordance that is an affordance for controlling a low-light capture mode when an amount of light in the field-of-view of the one or more cameras is below a second predetermined threshold provides a user with a quick and easy access to controlling the low-light capture mode when such control is likely to be needed and/or used. Reducing the number of inputs needed to perform an operation enhances the operability of the device and makes the user-device interface more efficient (e.g., by helping the user to provide proper inputs and reducing user mistakes when operating/interacting with the device) which, additionally, reduces power usage and improves battery life of the device by enabling the user to use the device more quickly and efficiently. In some embodiments, the electronic device (e.g., 600) receives a user input corresponding to the selection of the affordance (e.g. 650d) for controlling a low-light capture mode, and, in response to receiving the user input, the electronic device can change the state (e.g., active (e.g., on), inactive (e.g., off)) of the low-light capture mode and/or display a user interface to change the state of the low-light capture mode”),
performing a hands matting process in which attributes of the hand are identified using the image data stream (See Takemoto: Figs. 6-14, and [0051], “Further, the MR presentation system generates a finally corrected distance image 1420 by interpolating and extrapolating a partial area of the high reliability distance image 1410, if the area is defective compared to the original target object area included in the captured image 1401, within the region ranging to the contour line extracted from the captured image 1401. Through the above-mentioned processing flow, the MR presentation system corrects a distance measurement error of a moving target object using the captured image 1401 and the distance image 1405. An example method for generating the reliability image 1305, the high reliability distance image 1410, and the corrected distance image 1420 is described in detail below”; and [0109], “Next, in step S311, the virtual image generating unit 110 generates a virtual object image using three-dimensional model information of the virtual object and the corrected distance image stored in the storage unit 109. According to the example using the corrected distance image 1420 illustrated in FIG. 14, first, the virtual image generating unit 110 renders the three-dimensional model information of the virtual object and generates color information about the virtual object image together with the Z buffer value. In the present exemplary embodiment, it is presumed that the virtual object image is rendered in such a way as to have a resolution comparable to that of the captured image 1401. However, the resolution of the captured image is not necessarily equal to the resolution of the virtual object image. It is useful to apply scaling transformation to the captured image according to the resolution of the virtual object image”. Note that the various hand image generations may be mapped to the hands matting process in which the hands attributes, e.g., the contours of the hands or nails, are identified); and
blending the image data stream with virtual content (See Atlas: Figs. 3A-H, and Col. 10 Lines 53-67 ~ Col. 11 Lines 1-14, “FIG. 3A illustrates an example scene 302 depicting a real-world object 304. The scene may be viewed by a user of a head-mounted display (HMD) 100 such as that shown in FIG. 1. An image of the real-world object 304 and scene light 116 may be directed to an eye 120 of a user for viewing. The real-world object 304 may be displayed on the display device 110 in a perceived image 162 with particular attributes that are perceptible to the eye 120. For example, the real-world object 304 may be displayed in particular colors, such as hues of green, with particular brightness. The HMD 100 may overlay AR virtual content 157 on the scene light 116. Thus, the real-world object 304 may appear to the eye 120 as a background object, particularly if the AR virtual content 157 overlaps the real-world object 304 on the perceived image 162, in which case the AR virtual content 157 may appear to be blended with the overlapped area of the real-world object 304. In some cases, depending on the colors, brightness, or other visible attributes of the real-world object 304 and the AR virtual content 157, the AR virtual content 157 and/or the real-world object 304 may be difficult for the eye 120 to see and identify in the perceived image 162. For example, if there is a lack of contrast between the AR virtual content 157 and the real-world object 304, the eye 120 may have difficulty distinguishing the AR virtual content 157 and the real-world object 304 from each other. As another example, if the color of the AR virtual content 157 is darker, then little or no light 155 may be projected by the projector 112, and the AR virtual object 157 is effectively transparent to the eye 120”) in accordance with the attributes of the hand (See Yamashita: Figs. 27A-H, and [0162], “FIG. 27C shows a screen example in the case where the body image generated by the body shape extraction section 600 is processed such that the contour of the body image is made non-transparent and the inside of the body image is made semi-transparent, and then the processed body image is drawn over the display information generated by the display information generation section 700. Based on the above-described image composition, it is possible to perform an intuitive input operation and also it is easy to confirm the display information even during the operation”. Note that the hand image is blended into the virtual content of the GUI component, and the hand features, e.g., hand contours and fingers, are composited and displayed in the composited (blended) images as shown in Figs. 27A-H).
Regarding claim 14, Atlas, Marzari, Takemoto, and Yamashita teach all the features with respect to claim 13 as outlined above. Further, Atlas and Marzari teach that the method of claim 13, further comprising:
in accordance with a determination that that a second brightness value for a second portion of the image data stream fails to satisfy a first brightness threshold (See Atlas: Fig. 12, and Col. 15 Lines 8-24, “In particular embodiments, the computing device may determine that a visual enhancement is desired based on an assessment of whether the AR virtual content in the image, when displayed with the scene, would likely have poor perceptibility. The device may make such an assessment by comparing characteristics of the scene with characteristics with the image. For example, if the image having low additive contrast is an issue, and the underlying scene having high spatial variability is an issue, then the computing device may conclude that perceptibility may be an issue. Indeed, in some embodiments, one or more predetermined thresholds may be programmed into the computing device to selectively trigger when to perform adaptive rendering. For example, a look-ahead analysis of a particular scene may be performed to determine whether or not the potential perceptibility is favorable (e.g., independent of the image AR virtual content to be rendered)”. Note that the brightness level of the background object may be mapped to the brightness value of a first portion of the image),
disabling the hands matting process (See Marzari: Fig. 1-2, and [0042], “In accordance with some embodiments, a method is described. The method is performed at an electronic device having a display device and one or more cameras. The method comprises: receiving a request to display a camera user interface; and in response to receiving the request to display the camera user interface, displaying, via the display device, a camera user interface that includes: displaying, via the display device, a representation of a field-of-view of the one or more cameras; and in accordance with a determination that low-light conditions have been met, wherein the low-light conditions include a condition that is met when ambient light in the field-of-view of the one or more cameras is below a respective threshold, displaying, concurrently with the representation of the field-of-view of the one or more cameras, a control for adjusting a capture duration for capturing media in response to a request to capture media; and in accordance with a determination that the low-light conditions have not been met, forgoing display of the control for adjusting the capture duration”. Note that low-light condition not met may be mapped to fail to satisfy the first brightness threshold, and forgoing the adjustment control may be mapped to disable the hand matting process: Atlas teaches that the threshold is used to trigger whether adaptive rendering is employed or not, and Marzari teaches that if the low-light condition not met, adjustment of the image setting is disabled, thus, combining Atlas and Marzari will arrive at the scenario that the low-light condition not met, the adaptive rendering will be disabled, and the hand matting will be disabled).
Regarding claim 15, Atlas, Marzari, Takemoto, and Yamashita teach all the features with respect to claim 13 as outlined above. Further, Marzari and Yamashita teach that the method of claim 13, wherein the hands matting process is performed using a first hands matting model in accordance with the first brightness value satisfying the first brightness threshold (See Yamashita: Figs. 12-18, and [0116], “A second extraction method is a brightness threshold method. In the brightness threshold method, the body area is extracted by comparing the brightness value of each pixel of the image data outputted from the body shape input section 100 to a predetermined threshold”; and [0119], “When the extraction of the body area is completed, next, the body shape extraction section 600 extracts, based on the body area 607 extracted in step S604, a contour 609 of the extracted body area 607 as shown in FIG. 15 (step S606). This contour extraction process is performed by extracting, from the pixels included in the body area 607, pixels adjacent to the pixels of an area other than the body area 607. More specifically, from all of the pixels included in the body area 607, pixels each having four adjacent pixels above and below, to the left and right (or eight adjacent pixels further including the diagonally upper-right, diagonally upper-left, diagonally lower-right and diagonally lower-left pixels of the attention pixel) which include a pixel of an area other than the body area 607, are extracted. Note that a smoothing process may be performed on the extracted contour 609 as necessary. It is possible, by performing the smoothing process, to eliminate aliasing occurring in the contour 609”. Note that the hand contours may be mapped to the first hands matting model, and Fig. 16 with hands correction or transformation may be mapped to the second hands matting model), and further comprising:
in accordance with a determination that a second brightness value for a second portion of the image data stream fails to satisfy a first brightness threshold, performing the hands matting process on the image data stream using a second hands matting model (See Marzari: Figs. 5A-H, and [0322], “For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In this example, a contact with a characteristic intensity that does not exceed the first threshold results in a first operation, a contact with a characteristic intensity that exceeds the first intensity threshold and does not exceed the second intensity threshold results in a second operation, and a contact with a characteristic intensity that exceeds the second threshold results in a third operation. In some embodiments, a comparison between the characteristic intensity and one or more thresholds is used to determine whether or not to perform one or more operations (e.g., whether to perform a respective operation or forgo performing the respective operation), rather than being used to determine whether to perform a first operation or a second operation”. Note that it is logical to have three different operations if there are two thresholds: first operation if the intensity below the first threshold; second operation if the intensity is between the first and second thresholds; and third operation if the intensity is greater than the second threshold, assume the first threshold is less than the second threshold. In the same way, if there are four thresholds, then, it may have five different operations, and so on).
Claims 7 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Atlas, etc. (US 11887263 B1) in view of Marzari, etc. (US 20220053142 A1), further in view of Takemoto (US 20140140579 A1), Yamashita, etc. (US 20090167682 A1), and Stoecker, etc. (US 20060269111 A1).
Regarding claim 7, Atlas, Marzari, Takemoto, and Yamashita teach all the features with respect to claim 1 as outlined above. Further, Atlas, modified by Marzari, Takemoto, and Yamashita, fails to explicitly disclose that the non-transitory computer readable medium of claim 1, wherein the first brightness value for the first portion of the image data stream is determined based on a median luminance of a set of pixels of the first portion of the image data stream.
However, Stoecker teaches that the non-transitory computer readable medium of claim 1, wherein the first brightness value for the first portion of the image data stream is determined based on a median luminance of a set of pixels of the first portion of the image data stream (See Stoecker: Fig. 4, and [0115], “An estimation of hair widths is made. The digital hair removal algorithm hair mask width is iteratively dilated by a single unit using mathematical morphology until there is no significant change in the number of pixels below a median threshold, i.e., hair mask width is adequate, as a check on optimized hair finding using iterative morphologic dilation and the histogram of object sizes”).
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention was effectively filed to modify Atlas to have the non-transitory computer readable medium of claim 1, wherein the first brightness value for the first portion of the image data stream is determined based on a median luminance of a set of pixels of the first portion of the image data stream as taught by Stoecker in order to represent specific benign patterns for dysplastic nevi such as the peripheral ring pattern (dark network outside) and fried egg pattern (dark network or dark blotch in the center) as a vector (See Stoecker: Fig. 4, and [0245], “The advantage to this representation is that specific benign patterns for dysplastic nevi such as the peripheral ring pattern (dark network outside) and fried egg pattern (dark network or dark blotch in the center) can be represented as a vector. The peripheral ring pattern is often C-shaped, with that representation appearing in the vector components 1) and 4), showing a tan color and a scaled distance between centroids computed as a small fraction of a lesion radius in the outer two decile rings. Additional patterns, including malignant patterns such as central or peripheral scar-like depigmentation can be represented”). Atlas teaches a method and system that may adjust the brightness of the background image or the virtual content characteristics to make the blended virtual content clear or more visible based on some kinds of thresholds to trigger the adaptive renderings of the composite virtual content; while Stoecker teaches a system and method that may automatically detect features of the objects in the image using the median of pixel values as intensity threshold. Therefore, it is obvious to one of ordinary skill in the art to modify Atlas by Stoecker to have median pixel luminance value as the brightness threshold to trigger the hands matting process. The motivation to modify Atlas by Stoecker is “Use of known technique to improve similar devices (methods, or products) in the same way”.
Regarding claim 19, Atlas, Marzari, Takemoto, and Yamashita teach all the features with respect to claim 13 as outlined above. Further, Stoecker teaches that the method of claim 13, wherein the first brightness value for the first portion of the image data stream is determined based on a median luminance of a set of pixels of the first portion of the image data stream (See Stoecker: Fig. 4, and [0115], “An estimation of hair widths is made. The digital hair removal algorithm hair mask width is iteratively dilated by a single unit using mathematical morphology until there is no significant change in the number of pixels below a median threshold, i.e., hair mask width is adequate, as a check on optimized hair finding using iterative morphologic dilation and the histogram of object sizes”).
Allowable Subject Matter
Claims 4-6, 12, and 16-18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The best arts searched do not teach the cited limitations of “The non-transitory computer readable medium of claim 1, further comprising computer readable code to: in accordance with a determination that a second brightness value for a second portion of the image data stream fails to satisfy a second brightness threshold: determine a hand region in the second portion of the image data stream, determine a brightness of the hand region, and in accordance with a determination that the brightness in the hand region of the second portion of the image data stream fails to satisfy a third brightness threshold, disable the hands matting process.”
Claims 8 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The best arts searched do not teach the cited limitations of “The non-transitory computer readable medium of claim 1, wherein the computer readable code to perform the hands matting process comprises computer readable code to: select an alternative camera source in accordance with the first brightness value of the first portion of the image data stream; obtain additional image data from the alternative camera source; reproject the first portion of the image data stream to the additional image data; and identify attributes of the hand from the first portion of the image data stream in accordance with the reprojection.”
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GORDON G LIU whose telephone number is (571)270-0382. The examiner can normally be reached Monday - Friday 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona E Faulk can be reached at 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GORDON G LIU/Primary Examiner, Art Unit 2618