Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/03/2025 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
2.) Claim(s) 1-2 and 19-20 is/are rejected under 35 U.S.C. 102 (a1) (a2) as being anticipated by Koul et al. (US Pub No.: 2017/0286383A1).
With regard to Claim 1, Koul et al. disclose a method implemented on a computing device (Systems, apparatuses, services, platforms, and methods are discussed herein that provide assistance for user interface devices, Abstract; Paragraph 0013), the method comprising:
receiving an input from a user to initiate a camera of the computing device to capture an image of an environment (Turning first to FIG. 2A, assistance is provided to a user to capture an image. This assistance can include directing a user to move a camera or associated user device in a three-dimensional space to bring objects of interest into focus, into frame, into proper orientation, or to ensure desired features of an object of interest are able to be captured in an image. First, a user initiates capture of image or video of a scene (201) in assistance application 120. User device 110 can capture an image or video using camera 111 or other imaging equipment, Paragraphs 0031-0032; Figures 1-2C);
detecting one or more objects of interest in the environment (This assistance can include directing a user to move a camera or associated user device in a three-dimensional space to bring objects of interest into focus, into frame, into proper orientation, or to ensure desired features of an object of interest are able to be captured in an image, Paragraphs 0031-0032);
providing at least one of haptic, audio, or visual feedback to guide the user to position the computing device so that the one or more objects of interest become positioned within a viewfinder of the camera (This assistance can include directing a user to move a camera or associated user device in a three-dimensional space to bring objects of interest into focus, into frame, into proper orientation, or to ensure desired features of an object of interest are able to be captured in an image. The assistance can include directional prompts or alerts which direct a user to move an imaging device to better capture an image or element of interest in a scene. Directional notifications can prompt the user to move an imaging sensor of the imaging system of user device 110 (such as camera 111) to increase a recognition level of at least one element in the scene. The alerts can include audio, visual, tactile, or other alerts which can prompt directional positioning as well as capture initiation prompts to a user, such as prompting an alert indicating that the image is positioned and ready for capture, Paragraphs 0029-0031);
providing vocal feedback to identify the one or more objects of interest positioned within the viewfinder of the camera (Guidance can be provided to a visually impaired user to capture a particular object or to adequately frame an image about some objects of interest. This guidance can include a constant stream of description to the user to audibly indicate what is currently being captured by the image. Once a scene or associated objects are adequately arranged in an image, then the user can capture the image and potentially share via social media, text messaging, or other sharing services. This process can enable a visually impaired person or even an automated imaging system to take effective photographs using a digital imaging device, such as a smartphone or tablet computing device, Paragraphs 006-0070); and
causing the camera to capture the image (Once the desired criteria are met (203) then application 120 can instruct the user to finalize capture of the image (204). The instructions can comprise an audio instruction to the user. The audio instructions can include audio tones that change as a user brings objects of interest into frame and indicate when a desired object is properly positioned. The audio instructions can include spoken word instructions that direct the user to act accordingly, such as movement instructions. The instructions can also include haptic or vibration feedback to indicate to the user that objects are properly positioned. In some examples, the image can be automatically captured when a user has properly positioned camera 111 or properly positioned objects within a frame, Paragraph 0034).
Regarding Claim 2, Koul et al. disclose the method of claim 1, wherein the one or more objects of interest comprise a face of the user, and wherein the at least one of the haptic, audio, or visual feedback is provided to guide the user to center the face of the user in the image (Object recognition (i.e. identifying objects and where are they located in an image or scene), scene description services (i.e. generating plain language descriptions based on objects recognized in an image or scene). Intelligent human recognition and detection, such as face detection, gender detection, age estimation, and emotion recognition. Document boundary identification (i.e. edge detection, image centering) can also be applied to images to assist in centering or positioning documents or other elements within a frame of an image, Paragraph 0017).
With regard to Claim 19, Koul et al. disclose a computing device (Systems, apparatuses, services, platforms, and methods are discussed herein that provide assistance for user interface devices, Abstract; Paragraph 0013), the method comprising:
receive an input from a user to initiate a camera of the computing device to capture an image of an environment (Turning first to FIG. 2A, assistance is provided to a user to capture an image. This assistance can include directing a user to move a camera or associated user device in a three-dimensional space to bring objects of interest into focus, into frame, into proper orientation, or to ensure desired features of an object of interest are able to be captured in an image. First, a user initiates capture of image or video of a scene (201) in assistance application 120. User device 110 can capture an image or video using camera 111 or other imaging equipment, Paragraphs 0031-0032; Figures 1-2C);
detect one or more objects of interest in the environment (This assistance can include directing a user to move a camera or associated user device in a three-dimensional space to bring objects of interest into focus, into frame, into proper orientation, or to ensure desired features of an object of interest are able to be captured in an image, Paragraphs 0031-0032);
provide at least one of haptic, audio, or visual feedback to guide the user to position the computing device so that the one or more objects of interest become positioned within a viewfinder of the camera (This assistance can include directing a user to move a camera or associated user device in a three-dimensional space to bring objects of interest into focus, into frame, into proper orientation, or to ensure desired features of an object of interest are able to be captured in an image. The assistance can include directional prompts or alerts which direct a user to move an imaging device to better capture an image or element of interest in a scene. Directional notifications can prompt the user to move an imaging sensor of the imaging system of user device 110 (such as camera 111) to increase a recognition level of at least one element in the scene. The alerts can include audio, visual, tactile, or other alerts which can prompt directional positioning as well as capture initiation prompts to a user, such as prompting an alert indicating that the image is positioned and ready for capture, Paragraphs 0029-0031);
provide vocal feedback to identify the one or more objects of interest positioned within the viewfinder of the camera (Guidance can be provided to a visually impaired user to capture a particular object or to adequately frame an image about some objects of interest. This guidance can include a constant stream of description to the user to audibly indicate what is currently being captured by the image. Once a scene or associated objects are adequately arranged in an image, then the user can capture the image and potentially share via social media, text messaging, or other sharing services. This process can enable a visually impaired person or even an automated imaging system to take effective photographs using a digital imaging device, such as a smartphone or tablet computing device, Paragraphs 006-0070); and
cause the camera to capture the image (Once the desired criteria are met (203) then application 120 can instruct the user to finalize capture of the image (204). The instructions can comprise an audio instruction to the user. The audio instructions can include audio tones that change as a user brings objects of interest into frame and indicate when a desired object is properly positioned. The audio instructions can include spoken word instructions that direct the user to act accordingly, such as movement instructions. The instructions can also include haptic or vibration feedback to indicate to the user that objects are properly positioned. In some examples, the image can be automatically captured when a user has properly positioned camera 111 or properly positioned objects within a frame, Paragraph 0034).
In regard to computer program storing Claim 20, the claim corresponds to device claim 19 and method claim 1 and is rejected as discussed in the above rejection to claims 1 and 19.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3.) Claim(s) 3, 7-8 and 12-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Koul et al. (US Pub No.: 2017/0286383A1) as applied to claim 2 above, and further in view of Tartz et al. (US Pub No.: 2015/0334292A1).
In regard to Claim 3, Koul et al. do not explicitly disclose the method of claim 2, further comprising: determining a distance between a face portion of the image and a center of the image, wherein the at least one of the haptic, audio, or visual feedback is adjusted dynamically based on the distance. Tartz et al. disclose determining a distance between a face portion of the image and a center of the image, wherein the at least one of the haptic, audio, or visual feedback is adjusted dynamically based on the distance,
(Tartz et al. teach of methods, systems, computer-readable media, and apparatuses for providing haptic feedback to assist in capturing images. In some embodiments, a method for providing haptic feedback to assist in capturing images includes obtaining, via an image capture device, an ambient light measurement of an environment in which the image capture device is present. The method further includes detecting, via the image capture device, one or more objects within one or more image frames captured by the image capture device. The method also includes changing, via the image capture device, a manner in which haptic feedback is provided to a user of the image capture device, based at least in part on the obtained ambient light measurement and the detected one or more objects, Abstract of Tartz et al..
Tartz et al. teach that one or more objects within one or more image frames captured by the image capture device are detected. The detecting of the objects may be performed by an image capture module. For example, in FIG. 4A, the user's face within the image frame is detected by the image capture module. The image capture module may determine whether the object is within an optimal location and/or an optimal distance within the image frame, Paragraph 0099 of Tartz et al.. The manner in which the haptic feedback is provided is changed based at least in part on a location, within the one or more image frames, of the detected objects relative to an optimal location within the one or more image frames, Claims 1, 7 and 9 of Tartz et al.. When the user is attempting to position the device such that the user's face is in an optimal location with the image frame, the device may change a manner in which the haptic feedback is provided as the user's face gets closer to the center of the image frame (e.g., the device may vibrate more rapidly as the user's face gets closer to the center), Paragraphs 0005, 0048 of Tartz et al.. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to determine a distance between a face portion of the image and a center of the image, wherein the at least one of the haptic, audio, or visual feedback is adjusted dynamically based on the distance as taught by Tartz et al., because doing so helps provide the user with the assistance needed to continue to capture the correct image even when the content being captured is difficult to view on the display, Paragraphs 0003, 0047, 0050, 0093 of Tartz et al.).
Regarding Claim 7, Koul et al. do not explicitly disclose the method of claim 2, wherein at least one of an intensity or a duration of the haptic feedback is adjusted as the face approaches the center of the image. Tartz et al. teach of at least one of an intensity or a duration of the haptic feedback is adjusted as the face approaches the center of the image, (Tartz et al. teach of methods, systems, computer-readable media, and apparatuses for providing haptic feedback to assist in capturing images. In some embodiments, a method for providing haptic feedback to assist in capturing images includes obtaining, via an image capture device, an ambient light measurement of an environment in which the image capture device is present. The method further includes detecting, via the image capture device, one or more objects within one or more image frames captured by the image capture device. The method also includes changing, via the image capture device, a manner in which haptic feedback is provided to a user of the image capture device, based at least in part on the obtained ambient light measurement and the detected one or more objects, Abstract of Tartz et al.. The manner in which the haptic feedback is provided is changed based at least in part on a location, within the one or more image frames, of the detected objects relative to an optimal location within the one or more image frames, Claims 1, 7 and 9 of Tartz et al.. When the user is attempting to position the device such that the user's face is in an optimal location with the image frame, the device may change a manner in which the haptic feedback is provided as the user's face gets closer to the center of the image frame (e.g., the device may vibrate more rapidly as the user's face gets closer to the center), Paragraphs 0005, 0048 of Tartz et al.. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to have at least one of an intensity or a duration of the haptic feedback is adjusted as the face approaches the center of the image as taught by Tartz et al., because doing so helps provide the user with the assistance needed to continue to capture the correct image even when the content being captured is difficult to view on the display, Paragraphs 0003, 0047, 0050, 0093 of Tartz et al.).
With regard to Claim 8, Koul et al. do not explicitly disclose the method of claim 2, wherein the audio feedback comprises an indication of a direction to move the camera to position the face in the center of the image. Tartz et al. teach of audio feedback that comprises an indication of a direction to move the camera to position the face in the center of the image (Tartz et al. teach of methods, systems, computer-readable media, and apparatuses for providing haptic feedback to assist in capturing images. In some embodiments, a method for providing haptic feedback to assist in capturing images includes obtaining, via an image capture device, an ambient light measurement of an environment in which the image capture device is present. The method further includes detecting, via the image capture device, one or more objects within one or more image frames captured by the image capture device. The method also includes changing, via the image capture device, a manner in which haptic feedback is provided to a user of the image capture device, based at least in part on the obtained ambient light measurement and the detected one or more objects, Abstract of Tartz et al..
The manner in which the haptic feedback is provided is changed based at least in part on a location, within the one or more image frames, of the detected objects relative to an optimal location within the one or more image frames, Claims 1, 7 and 9 of Tartz et al.. When the user is attempting to position the device such that the user's face is in an optimal location with the image frame, the device may change a manner in which the haptic feedback is provided as the user's face gets closer to the center of the image frame (e.g., the device may vibrate more rapidly as the user's face gets closer to the center), Paragraphs 0005, 0048 of Tartz et al.. The communication device 100 may provide audio feedback 450 concurrently with the vibration based haptic feedback 430. For example, the communication device 100 may play an audio phrase, via speaker 150, Paragraphs 0075, 0090-0091; Figures 7a-7b of Tartz et al.. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to have audio feedback that comprises an indication of a direction to move the camera to position the face in the center of the image as taught by Tartz et al., because doing so helps provide the user with the assistance needed to continue to capture the correct image even when the content being captured is difficult to view on the display, Paragraphs 0003, 0047, 0050, 0093 of Tartz et al.).
Regarding Claim 12, Koul et al. do not explicitly disclose the method of claim 2, wherein capturing of the image is performed automatically when the face reaches the center of the image. Tartz et al. teach of capturing of the image is performed automatically when the face reaches the center of the image, (Tartz et al. teach of methods, systems, computer-readable media, and apparatuses for providing haptic feedback to assist in capturing images. In some embodiments, a method for providing haptic feedback to assist in capturing images includes obtaining, via an image capture device, an ambient light measurement of an environment in which the image capture device is present. The method further includes detecting, via the image capture device, one or more objects within one or more image frames captured by the image capture device. The method also includes changing, via the image capture device, a manner in which haptic feedback is provided to a user of the image capture device, based at least in part on the obtained ambient light measurement and the detected one or more objects, Abstract of Tartz et al..
The image capture module 194 (FIG. 1) may automatically capture and store an image upon detecting the face 410 within the image frame of the camera 186. There may be a delay, e.g. 2 seconds, between detecting the frame and capturing the image. The communication device 100 may provide vibration based haptic feedback 430 once the image has been captured, Paragraph 0069 of Tartz et al.. When the user is attempting to position the device such that the user's face is in an optimal location with the image frame, the device may change a manner in which the haptic feedback is provided as the user's face gets closer to the center of the image frame (e.g., the device may vibrate more rapidly as the user's face gets closer to the center), Paragraphs 0005, 0048 of Tartz et al..
The communication device 100 may provide audio feedback 450 concurrently with the vibration based haptic feedback 430. For example, the communication device 100 may play an audio phrase, via speaker 150, Paragraphs 0075, 0090-0091; Figures 7a-7b of Tartz et al.. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to capture the image automatically when the face reaches the center of the image as taught by Tartz et al., because by automatically capturing the image, the device provides assistance to the user in capturing images while in bright environments where the content on the display may be difficult to view, Paragraphs 0003, 0047, 0050, 0069, 0093 of Tartz et al.).
In regard to Claim 13, Koul et al. do not explicitly disclose the method of claim 1, wherein providing the vocal feedback to identify the one or more objects of interest positioned within the viewfinder of the camera is performed after the one or more objects of interest are positioned within the viewfinder of the camera for a threshold period of time. Tartz et al. teach that providing the vocal feedback to identify the one or more objects of interest positioned within the viewfinder of the camera is performed after the one or more objects of interest are positioned within the viewfinder of the camera for a threshold period of time, (Tartz et al. teach of methods, systems, computer-readable media, and apparatuses for providing haptic feedback to assist in capturing images. In some embodiments, a method for providing haptic feedback to assist in capturing images includes obtaining, via an image capture device, an ambient light measurement of an environment in which the image capture device is present. The method further includes detecting, via the image capture device, one or more objects within one or more image frames captured by the image capture device. The method also includes changing, via the image capture device, a manner in which haptic feedback is provided to a user of the image capture device, based at least in part on the obtained ambient light measurement and the detected one or more objects, Abstract of Tartz et al..
The image capture module 194 (FIG. 1) may automatically capture and store an image upon detecting the face 410 within the image frame of the camera 186. There may be a delay, e.g. 2 seconds, between detecting the frame and capturing the image. The communication device 100 may provide vibration based haptic feedback 430 once the image has been captured. The communication device 100 may provide audio feedback 450 concurrently with the vibration based haptic feedback 430. For example, the communication device 100 may play an audio phrase, via speaker 150, reciting “Face detected within the image frame” concurrently with the vibration based haptic feedback 430. In some embodiments, the communication device 100 may provide text feedback 440 concurrently with the vibration based haptic feedback 430, Paragraphs 0069-0070 of Tartz et al..
The communication device 100 may provide audio feedback 450 concurrently with the vibration based haptic feedback 430. For example, the communication device 100 may play an audio phrase, via speaker 150, Paragraphs 0075, 0090-0091; Figures 7a-7b of Tartz et al.. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to provide the vocal feedback to identify the one or more objects of interest positioned within the viewfinder of the camera after the one or more objects of interest are positioned within the viewfinder of the camera for a threshold period of time as taught by Tartz et al., because doing so provides assistance and adequate time needed for the user in capturing images while in bright environments where the content on the display may be difficult to view, Paragraphs 0003, 0047, 0050, 0069, 0093 of Tartz et al.).
Regarding Claim 14, Koul et al. disclose the method of claim 1, wherein capturing the image is performed after the one or more objects of interest are positioned within the viewfinder of the camera for a threshold period of time. Tartz et al. teach of capturing the image after the one or more objects of interest are positioned within the viewfinder of the camera for a threshold period of time, (Tartz et al. teach of methods, systems, computer-readable media, and apparatuses for providing haptic feedback to assist in capturing images. In some embodiments, a method for providing haptic feedback to assist in capturing images includes obtaining, via an image capture device, an ambient light measurement of an environment in which the image capture device is present. The method further includes detecting, via the image capture device, one or more objects within one or more image frames captured by the image capture device. The method also includes changing, via the image capture device, a manner in which haptic feedback is provided to a user of the image capture device, based at least in part on the obtained ambient light measurement and the detected one or more objects, Abstract of Tartz et al..
The image capture module 194 (FIG. 1) may automatically capture and store an image upon detecting the face 410 within the image frame of the camera 186. There may be a delay, e.g. 2 seconds, between detecting the frame and capturing the image. The communication device 100 may provide vibration based haptic feedback 430 once the image has been captured. The communication device 100 may provide audio feedback 450 concurrently with the vibration based haptic feedback 430. For example, the communication device 100 may play an audio phrase, via speaker 150, reciting “Face detected within the image frame” concurrently with the vibration based haptic feedback 430. In some embodiments, the communication device 100 may provide text feedback 440 concurrently with the vibration based haptic feedback 430, Paragraphs 0069-0070 of Tartz et al..
The communication device 100 may provide audio feedback 450 concurrently with the vibration based haptic feedback 430. For example, the communication device 100 may play an audio phrase, via speaker 150, Paragraphs 0075, 0090-0091; Figures 7a-7b of Tartz et al.. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to capture the image after the one or more objects of interest are positioned within the viewfinder of the camera for a threshold period of time as taught by Tartz et al., because doing so provides assistance and adequate time needed for the user in capturing images while in bright environments where the content on the display may be difficult to view, Paragraphs 0003, 0047, 0050, 0069, 0093 of Tartz et al.).
4.) Claim(s) 4 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Koul et al. (US Pub No.: 2017/0286383A1) as applied to claims 1-2 above, and further in view of Holland et al. (US Pub No.: 2012/0327258A1).
Regarding Claim 4, Koul et al. do not explicitly disclose the method of claim 2, further comprising dividing the viewfinder into a plurality of zones, wherein the at least one of the haptic, audio, or visual feedback is adjusted responsive to transitions between respective zones of the plurality of zones. Holland et al. teach of dividing a viewfinder into a plurality of zones (plurality of regions), wherein the at least one of the haptic, audio, or visual feedback is adjusted responsive to transitions between respective zones of the plurality of zones,
(Holland et al. teach of techniques and mechanisms that are provided for facilitating the capture and review of visual images by visually impaired users. In one implementation, these techniques and mechanisms provide pre image capture functionality and captured image review functionality. With the pre image capture functionality, audio messages are provided to the user to help the user position an image capturing mechanism properly to capture the desired subjects in a picture, to frame the subjects properly within the picture, to size the subjects properly within the picture, etc. With the image review functionality, audio messages are provided to the user to help the user enjoy and "visualize" a visual image that has been captured and is being displayed to the user. With these functionalities, a visually impaired user is able capture and review images to a much greater degree than is currently possible, Abstract and Figure 5 of Holland et al..
Holland et al. teach that nine possibilities may be represented as nine regions, as shown in FIG. 5. These regions (A through I) correspond to the various regions of a frame. In one embodiment, if any portion of a face is within region A of the frame, the device 100 concludes that the face is close to the top-left edge of the frame. If any portion of a face is within region C of the frame, the device 100 concludes that the face is close to the top-right edge of the frame. If any portion of a face is within region G of the frame, the device 100 concludes that the face is close to the bottom-left edge of the frame, and if any portion of a face is within region I of the frame, the device 100 concludes that the face is close to the bottom-right edge of the frame. The device 100 may also provide an audio message regarding the location of the faces within the frame. For example, if a face is too close to the left edge of the frame, the device 100 may provide an audio message that says "face near left edge". If a face is too close to both the left edge and the top edge of the frame, the device 100 may provide an audio message that says "face near top-left edge". Similar audio messages may be provided if a face is too close to the top edge, right edge, bottom edge, top-right edge, bottom-left edge, or the bottom-right edge of the frame (see FIG. 5 for the various regions of the frame). If the faces are centered within the frame, the device 100 may provide an audio message that says "faces centered" (or provide no positioning message at all), Paragraphs 0058-0067; Figure 5 of Holland et al.. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to divide a viewfinder into a plurality of zones (plurality of regions), wherein the at least one of the haptic, audio, or visual feedback is adjusted responsive to transitions between respective zones of the plurality of zones as taught by Holland et al., because the positional analysis helps in determining what type of subsequent processing to perform for optimal imaging of a face).
Regarding Claim 15, Koul et al. do not explicitly disclose the method of claim 1, wherein the one or more objects of interest are identified from a predefined allowed list of objects. Holland et al. teach of identifying one or more objects of interest from a predefined allowed list of objects (from a database),
(Holland et al. teach of techniques and mechanisms that are provided for facilitating the capture and review of visual images by visually impaired users. In one implementation, these techniques and mechanisms provide pre image capture functionality and captured image review functionality. With the pre image capture functionality, audio messages are provided to the user to help the user position an image capturing mechanism properly to capture the desired subjects in a picture, to frame the subjects properly within the picture, to size the subjects properly within the picture, etc. With the image review functionality, audio messages are provided to the user to help the user enjoy and "visualize" a visual image that has been captured and is being displayed to the user. With these functionalities, a visually impaired user is able capture and review images to a much greater degree than is currently possible, Abstract and Figure 5 of Holland et al..
Holland et al. teach that with facial recognition, each face detected in the frame is analyzed to determine whether the face belongs to a person that has already been identified to the device 100. Facial recognition may involve deriving, for each detected face, some facial characteristic values. These facial characteristic values may then be compared with facial characteristic values of known, identified faces in a database to determine whether the detected face is close enough to an identified face to conclude that the detected face belongs to the same person as the identified face. If a conclusion is made that a detected face belongs to the same person as an identified face, then the identity or name associated with the identified face may be associated with the detected face, Paragraph 0039 of Holland et al.. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to identify one or more objects of interest from a predefined allowed list of objects (from a database) as taught by Holland et al., because this is a well-known way of focusing in on areas and object of interest that are important to a user).
5.) Claim(s) 9-11 and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Koul et al. (US Pub No.: 2017/0286383A1) as applied to claims 1-2 above, and further in view of Vetter (US Pub No.: 2017/0236298A1).
In regard to Claim 9, Koul does not explicitly disclose the method of claim 2, wherein the visual feedback comprises portions of a circle illuminated to indicate a direction to move the camera to position the face in the center of the image. Vetter teach of visual feedback comprises portions of a circle illuminated to indicate a direction to move the camera to position the face in the center of the image, (Vetter discloses a device and a method for determining a position of a user's body portion. The device includes a camera, configured to capture the body portion, and a display for providing visual feedback. A sensor determines at least one of a roll angle, a pitch angle, and a yaw angle of the device, and an interface receives picture data related to a pictorial representation of the body portion captured and sensor data related to the determined angle of the device. An analyzer analyzes, based on the picture data, whether the captured body portion is within a predetermined region of the picture and, based on the sensor data, whether at least one of the roll angle, the pitch angle, and the yaw angle is within a predetermined angle range. The method includes capturing the body portion, providing visual feedback, receiving angle data, receiving picture data, and analyzing whether the captured body portion is within a predetermined region and whether at least one of the roll, pitch, and yaw angles is within a predetermined angle range, Abstract of Vetter.
Vetter teaches of a face detection algorithm enables the analyzer 9 to detect the face of the user 3 within the pictorial representation 5 captured by the camera 4. The previously discussed first circle 33 is overlayed onto the pictorial representation 5 of the user 3 such that the first circle 33 substantially covers the detected face of the user 3. The algorithm is further configured to track the face of the user 3, once it has been detected, preferably in real-time. Accordingly, if the user 3 moves his face relative to the camera 4, the algorithm is configured to continuously detect the user's face so as to follow the user's movements. an image 46 of three consecutive arrow heads pointing into the direction into which the user 3 shall move his/her face. Accordingly, this image 46 (e.g. arrow heads) is being displayed on the display 6 in order to provide visual feedback to the user 3 informing him/her in which direction he/she shall move so as to move his/her body portion inside the predetermined region 33 of the picture captured by the camera 4. The image 46 may point into the direction of the center of the second circle 34, Paragraphs 0057, 0085-0092; Figures 4a-4c of Vetter. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul to visual feedback comprises portions of a circle illuminated to indicate a direction to move the camera to position the face in the center of the image as taught by Vetter, because it helps accurately guide a user to position the face for proper capture by the camera).
With regard to Claim 10, Koul does not explicitly disclose the method of claim 2, wherein the visual feedback comprises a high-contrast outline surrounding the user or the face of the user. Vetter discloses a visual feedback that comprises a high-contrast outline surrounding the user or the face of the user (see circles surrounding/outlining user or user’s face), (Vetter discloses a device and a method for determining a position of a user's body portion. The device includes a camera, configured to capture the body portion, and a display for providing visual feedback. A sensor determines at least one of a roll angle, a pitch angle, and a yaw angle of the device, and an interface receives picture data related to a pictorial representation of the body portion captured and sensor data related to the determined angle of the device. An analyzer analyzes, based on the picture data, whether the captured body portion is within a predetermined region of the picture and, based on the sensor data, whether at least one of the roll angle, the pitch angle, and the yaw angle is within a predetermined angle range. The method includes capturing the body portion, providing visual feedback, receiving angle data, receiving picture data, and analyzing whether the captured body portion is within a predetermined region and whether at least one of the roll, pitch, and yaw angles is within a predetermined angle range, Abstract of Vetter.
Vetter teaches of a face detection algorithm enables the analyzer 9 to detect the face of the user 3 within the pictorial representation 5 captured by the camera 4. The previously discussed first circle 33 is overlayed onto the pictorial representation 5 of the user 3 such that the first circle 33 substantially covers the detected face of the user 3. The algorithm is further configured to track the face of the user 3, once it has been detected, preferably in real-time. Accordingly, if the user 3 moves his face relative to the camera 4, the algorithm is configured to continuously detect the user's face so as to follow the user's movements. an image 46 of three consecutive arrow heads pointing into the direction into which the user 3 shall move his/her face. Accordingly, this image 46 (e.g. arrow heads) is being displayed on the display 6 in order to provide visual feedback to the user 3 informing him/her in which direction he/she shall move so as to move his/her body portion inside the predetermined region 33 of the picture captured by the camera 4. The image 46 may point into the direction of the center of the second circle 34, Paragraphs 0057, 0085-0092; Figures 4a-4c of Vetter. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul to include visual feedback that comprises a high-contrast outline surrounding the user or the face of the user (see circles surrounding/outlining user or user’s face), because it helps accurately guide a user to position the face for proper capture by the camera).
Regarding Claim 11, Koul et al. and Vetter disclose the method of claim 10, wherein brightness of the high-contrast outline increases as the face approaches the center of the image (As can be seen, if the detected body portion, i.e. the user's face in this example, is aligned with the alignment guides 50, then the second circle 34 may change its appearance, e.g. by switching to a different color or contrast. In this example, the second circle 34 may switch to a darker contrast, Paragraph 0115 of Vetter. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to have brightness of the high-contrast outline increase as the face approaches the center of the image as taught by Vetter, because it helps accurately guide a user to position the face for proper capture by the camera).
In regard to Claim 17, Koul et al. do not explicitly disclose the method of claim 1, wherein the visual feedback comprises a shape surrounding an object of interest that changes color depending on whether the object (i) is partially cropped at a border of the viewfinder, (ii) is fully contained within the viewfinder but not at a center of the image, or (iii) is located at the center of the image. Vetter teaches of visual feedback that comprises a shape surrounding an object of interest (circle surrounding object of interest) that changes color depending on whether the object (i) is partially cropped at a border of the viewfinder, (ii) is fully contained within the viewfinder but not at a center of the image, or (iii) is located at the center of the image, (Vetter discloses a device and a method for determining a position of a user's body portion. The device includes a camera, configured to capture the body portion, and a display for providing visual feedback. A sensor determines at least one of a roll angle, a pitch angle, and a yaw angle of the device, and an interface receives picture data related to a pictorial representation of the body portion captured and sensor data related to the determined angle of the device. An analyzer analyzes, based on the picture data, whether the captured body portion is within a predetermined region of the picture and, based on the sensor data, whether at least one of the roll angle, the pitch angle, and the yaw angle is within a predetermined angle range. The method includes capturing the body portion, providing visual feedback, receiving angle data, receiving picture data, and analyzing whether the captured body portion is within a predetermined region and whether at least one of the roll, pitch, and yaw angles is within a predetermined angle range, Abstract of Vetter. As can be seen, if the detected body portion, i.e. the user's face in this example, is aligned with the alignment guides 50, then the second circle 34 may change its appearance, e.g. by switching to a different color or contrast. In this example, the second circle 34 may switch to a darker contrast, Paragraph 0115 of Vetter. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to have visual feedback that comprises a shape surrounding an object of interest (circle surrounding object of interest) that changes color depending on whether the object (i) is partially cropped at a border of the viewfinder, (ii) is fully contained within the viewfinder but not at a center of the image, or (iii) is located at the center of the image as taught by Vetter, because it helps accurately guide a user to position the face for proper capture by the camera).
With regard to Claim 18, Koul et al. do not explicitly disclose the method of claim 1, wherein the visual feedback comprises border guidance highlighting one or more borders of the viewfinder which are cropping one or more objects of interest. Vetter discloses a visual feedback that comprises border guidance highlighting one or more borders of the viewfinder which are cropping one or more objects of interest (Vetter discloses a device and a method for determining a position of a user's body portion. The device includes a camera, configured to capture the body portion, and a display for providing visual feedback. A sensor determines at least one of a roll angle, a pitch angle, and a yaw angle of the device, and an interface receives picture data related to a pictorial representation of the body portion captured and sensor data related to the determined angle of the device. An analyzer analyzes, based on the picture data, whether the captured body portion is within a predetermined region of the picture and, based on the sensor data, whether at least one of the roll angle, the pitch angle, and the yaw angle is within a predetermined angle range. The method includes capturing the body portion, providing visual feedback, receiving angle data, receiving picture data, and analyzing whether the captured body portion is within a predetermined region and whether at least one of the roll, pitch, and yaw angles is within a predetermined angle range, Abstract of Vetter. As can be seen, if the detected body portion, i.e. the user's face in this example, is aligned with the alignment guides 50, then the second circle 34 may change its appearance, e.g. by switching to a different color or contrast. In this example, the second circle 34 may switch to a darker contrast, Paragraph 0115 of Vetter. It would have been obvious and well-known to one of ordinary skill in the art before the effective filing date of the claimed invention to enable the teachings of Koul et al. to have a visual feedback that comprises border guidance highlighting one or more borders of the viewfinder which are cropping one or more objects of interest as taught by Vetter, because it helps accurately guide a user to position the face for proper capture by the camera).
6.) Allowable Subject Matter
Claims 5-6 and 16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PRITHAM DAVID PRABHAKHER whose telephone number is (571)270-1128. The examiner can normally be reached Monday to Friday 8:00 am to 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lin Ye can be reached at 5712727372. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Pritham David Prabhakher
Patent Examiner
Pritham.Prabhakher@uspto.gov
/PRITHAM D PRABHAKHER/ Primary Examiner, Art Unit 2638
