DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5, 7-11, 13, and 17-20 is/are rejected under 35 U.S.C. 103(a) as being obvious over Karakotsios (2016/0283778) hereinafter, Karakotsios in view of Guo et al (2016/0183014), further in view of Adsumilli et al (2017/0006220) hereinafter, Adsumilli.
In regards to claim 1, a Karakotsios teaches method comprising (abstract):
obtaining first sensor data from a first device worn on a head (fig. 3a (322) [0027);
obtaining second sensor data from a second device worn on the head, wherein the first sensor data and the second sensor data correspond to a first time [0027-032]; ;[0018, 0021, 0030-0032] (fig. 2 (202) and 206)) and
[0027] As illustrated in FIG. 3(b), there can be at least one camera 322 on the front of the glasses as well. This camera can be used to capture image information about objects that might be within the user's line of sight. The image information can be captured and analyzed to determine the relative direction of one or more objects, which can be used with the gaze direction information to determine the object at which the user is gazing. In some embodiments, stereoscopic cameras or multiple cameras can be used to determine three-dimensional information about the relative positions of the objects with respect to the glasses, but in general the proximity of the glasses to the user's eyes means that the point of view will be substantially similar, such that a direction to an object from a user's eyes will be almost the same as the direction from the glasses to that object. Thus, processing needs can be reduced while using the glasses. Further, as discussed, such a device enables the gaze to be determined even when the user is turning his or her head, or might otherwise not be facing a computing device.
[0030] In some embodiments where high resolution and/or precision might not be needed, the glasses might utilize only one or more front-facing cameras in order to identify an object, or at least narrow the search field. For example, a user might turn his or her head to the left or the right, which can give a general indication of the direction in which the user is looking. The user also can be trained to face an object of interest in such a mode if the viewing direction is to be interpreted as input, instead of just moving the user's eyes while keeping the head substantially stationary. To save power in some embodiments, a pair of glasses might first attempt to determine an object of interest based on the direction the glasses are pointing, and then activate the gaze determination components if the object cannot be determined with at least an acceptable amount of certainty. In embodiments where the direction of the glasses is used to identify objects, information from sensors in the glasses, such as position sensors (e.g., GPS), motion sensors (e.g., gyroscopes, accelerometers, etc.), and the like, can be used to determine the relative position, orientation, and/or direction of the glasses. Various other approaches can be used as well within the scope of the various embodiments.
PNG
media_image1.png
672
448
media_image1.png
Greyscale
PNG
media_image2.png
678
440
media_image2.png
Greyscale
in response to receiving a selection input, performing a selection of the object [023-0025, 0032-0033,0046] “In some embodiments, a user gazing at a device to select the device can provide an audible command, perform a gesture, provide a motion, or indicate another such input that can be used to control an aspect of the device. For example, a user can say “on” to activate the device, might make an “up” motion with a hand to increase the volume, or might squeeze a pressure-sensitive portion of the device to change to the next “favorite” station, among other such actions.”[0024] (fig. 4 402-412))
PNG
media_image3.png
730
582
media_image3.png
Greyscale
Karakotsios fails to expressly teach a first earn worn device worn by a user; a second ear worn device worn by the user.
However, Guo teaches a first earn worn device worn by a user; a second ear worn device worn by the user.[0021, 0026, 0031, 0061-0076],(fig. 2 and 4 (14 imaging device and fig. 5 (detecting an object)) Guo
1
PNG
media_image4.png
558
702
media_image4.png
Greyscale
[0021] The image capture device may be positioned in a housing of the hearing device, and the hearing device is configured to be positioned at an ear of the user. This allows detection of e.g. movement of the head and/or capturing images from a well-defined direction relative to the head of the user. The image capture device may alternatively be positioned remote from a housing of the hearing device and the image capture device may then be in wired or wireless communication with the processor. Memory devices may be included for buffering and/or storing images. A remote location may be needed if images are to be captured in the direction of the face of the user and the hearing device is positioned at least partly in the ear canal of the user. This may e.g. be achieved via an image capture device mounted in or on a set of glasses or in a body-worn housing, e.g. for being worn at the chest of the user or a device located on the pinna and either in wired or wireless communication with the in-ear part. Multiple image capture devices may be combined, e.g. to ensure a more complete field of view. Also, image capture devices facing different directions. As an example of multiple image capture device may be one or more looking forward and one or more looking backwards.
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios to further include a first earn worn device worn by a user; a second ear worn device worn by the user as taught by Guo in order to allow for a smaller packaged device that doesn’t obscure the view of the user. Examiner notes Guo, like Karakotsios, teaches using either a glasses form or ear buds [0031].
Karakotsios and Guo fail to teach identifying an object in an overlapping region of the first sensor data and the second sensor data.
However, Adsumilli teaches identifying an object in an overlapping region of the first sensor data and the second sensor data.(fig. 1a (105a and 105b )(fig. 6 (602-606)(fig. 5a (510))
PNG
media_image5.png
774
756
media_image5.png
Greyscale
PNG
media_image6.png
830
632
media_image6.png
Greyscale
PNG
media_image7.png
594
558
media_image7.png
Greyscale
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios and Guo to further include identifying an object in an overlapping region of the first sensor data and the second sensor data as taught by Adsumilli in order to aid in the selection and calibration of multi-camera scenario [0036]
In regards to claim 9, Karakotsios teaches a non-transitory computer readable medium comprising computer readable code executable by one or more processor to (fig. 8 (802-812)), : obtain first sensor data from a first device worn on a head (fig. 3a (322) [0027-0032]);; obtain second sensor data from a second device worn on the head, wherein the first sensor data and the second sensor data correspond to a first time; [0027-032]; [0018, 0021, 0030] (fig. 2 (202) and 206)); and in response to receiving a selection input, perform selection operation on the object [0024-0025,0033, 0046] (fig. 4 402-412))
Karakotsios fails to expressly teach a first earn worn device worn by a user; a second ear worn device worn by the user.
However, Guo teaches a first earn worn device worn by a user; a second ear worn device worn by the user.[0021, 0026, 0031, 0061-0076],(fig. 2 and 4 (14 imaging device and fig. 5 (detecting an object))fig. 8 (14)).
PNG
media_image8.png
582
836
media_image8.png
Greyscale
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios to further include a first earn worn device worn by a user; a second ear worn device worn by the user as taught by Guo in order to allow for a smaller packaged device that doesn’t obscure the view of the user. Examiner notes Guo teaches using a glass form or ear buds [0031].
Karakotsios and Guo fail to teach identify an object in an overlapping region of the first sensor data and the second sensor data.
However, Adsumilli teaches identify an object in an overlapping region of the first sensor data and the second sensor data.(fig. 6 (602-606)(fig. 5a (510))
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios and Guo to further include identifying an object in an overlapping region of the first sensor data and the second sensor data as taught by Adsumilli in order to aid in the selection and calibration of multi-camera scenario [0036]
In regards to claim 17, Karakotsios teaches comprising: one or more processors; and one or more computer readable media comprising computer readable code executable by the one or more processors to (fig. 8 (802-812)): obtain first sensor data from a first device worn on a head (fig. 3a (322) [0027-0032);; obtain second sensor data from a second device worn on the head, wherein the first sensor data and the second sensor data correspond to a first time[0027-032]; determine a head position for the head based on the first sensor data and the second sensor data; ;[0018, 0021, 0030] (fig. 2 (202) and 206)) and in response to receiving a selection input, perform a selection operation on the object. [0024-0025,0033 0046] ] (fig. 4 402-412))
Karakotsios fails to expressly teach a first earn worn device worn by a user; a second ear worn device worn by the user.
However, Guo teaches a first earn worn device worn by a user; a second ear worn device worn by the user.[0021, 0026, 0031, 0061-0076],(fig. 2 and 4 (14 imaging device and fig. 5 (detecting an object))
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios to further include a first earn worn device worn by a user; a second ear worn device worn by the user as taught by Guo in order to allow for a smaller packaged device that doesn’t obscure the view of the user. Examiner notes Guo teaches using a glass form or ear buds [0031].
Karakotsios and Guo fail to teach identify an object in an overlapping region of the first sensor data and the second sensor data.
However, Adsumilli teaches identify an object in an overlapping region of the first sensor data and the second sensor data.(fig. 6 (602-606)(fig. 5a (510))
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios and Guo to further include identify an object in an overlapping region of the first sensor data and the second sensor data as taught by Adsumilli in order to aid in the selection and calibration of multi-camera scenario [0036]
In regards to claim 2, Karakotsios in view of Guo and Adsumilli teaches method of claim 1, wherein the first sensor data comprises first orientation data, and wherein the second sensor data comprises second orientation data [0030-0032]. Karakotsios and [0032] Guo
In regards to claim 3, Karakotsios in view of Guo and Adsumilli teaches method of claim 1, wherein the first sensor data comprises first image data, wherein the second sensor data further comprises second image data, and determinea direction of the head based on an overlap between the first image data and the second image data [0027-0032] stereoscopic cameras must have overlap. Karakotsios
In regards to claim 5, Karakotsios in view of Guo and Adsumilli teaches method of claim 1, wherein the selection input comprises a selection gesture [0024,0033, 0043]. Karakotsios
In regards to claim 7, Karakotsios in view of Guo and Adsumilli teaches the method of claim 1, wherein the selection input is detected on an accessory device, and wherein the selection input is received from the accessory device [0033,0043, 0051, 0062] touch on screen. Karakotsios
In regards to claim 8, Karakotsios in view of Guo and Adsumilli teaches wherein the first device comprises a first earbud [0031] Karakotsios and wherein the second device comprises a second earbud of the earbud pair (fig. 2 and fig. 4 14 )Guo).
In regards to claim 10, Karakotsios in view of Guo and Adsumilli teaches non-transitory computer readable medium of claim 9, wherein the first sensor data comprises first orientation data, and wherein the second sensor data comprises second orientation data [0027,0031,0032,0042,0049,0052] . Karakotsios and [0021, 0026, 0031-0032, 0061-0076] Guo.
In regards to claim 11, Karakotsios in view of Guo and Adsumilli teaches non-transitory computer readable medium of claim 9, wherein the first sensor data comprises first image data, further comprising computer readable code wherein the second sensor data comprises second image data, and determine a direction of the head based on an overlap between the first image data and the second image data. [0027-0032] stereoscopic cameras must have overlap. Karakotsios
In regards to claim 13, Karakotsios in view of Guo and Adsumilli teaches non-transitory computer readable medium of claim 9, wherein the selection input comprises a selection gesture. [0024,0033, 0043]. Karakotsios
In regards to claim 18, Karakotsios in view of Guo and Adsumilli teaches system of claim 17, wherein the one or more processors and the one or more computer readable media are comprised in a mobile device communicably coupled to an audio headset comprising the first device and the second device [0031]. Karakotsios
In regards to claim 19, Karakotsios in view of Guo and Adsumilli teaches system of claim 17, wherein the first sensor data comprises first orientation data, and wherein the second sensor data comprises second orientation data. [0027,0031,0032,0042,0049,0052] . Karakotsios
In regards to claim 20, Karakotsios in view of Guo and Adsumilli teaches system of claim 17, wherein the first sensor data comprises first image data, wherein the second sensor data comprises second image data, comprising computer readable code to and determine a direction of the head based on an overlap between the first image data and the second image data. [0027-0032] stereoscopic cameras must have overlap. Karakotsios
Claim(s) 6 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Karakotsios and Guo and Adsumilli in view of Mekenkamp (2009/0251559) hereinafter, Mekenkamp.
In regards to claim 6, Karakotsios and Guo and Adsumilli fails to teach the method of claim 1, further comprising: generating audio feedback corresponding to identifying the object. Examiner notes Guo appears to provide haptic feedback [0035]
However, Mekenkamp teaches further comprising: generating audio feedback corresponding to identifying the object.[0008].
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios and Guo to further include teaches further comprising: generating audio feedback regarding the object in response to identifying the object to provide feedback to the user when the object is usable [0008].
In regards to claim 14, Karakotsios and Guo and Adsumilli in view of Mekenkamp teaches the non-transitory computer readable medium of claim 9,see rational of claim 6, further comprising computer readable code to: generate audio feedback corresponding in response to identifying the object [0008] Mekenkamp.
Allowable Subject Matter
Claims 15-16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-3,5-11, and13-20 have been considered but are moot because the new ground of rejection does not rely on any combination of references applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant contends:
PNG
media_image9.png
248
778
media_image9.png
Greyscale
PNG
media_image10.png
120
706
media_image10.png
Greyscale
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Examiner notes newly cited reference Adsumilli teaches identify an object in an overlapping region of the first sensor data and the second sensor data.(fig. 6 (602-606)(fig. 5a (510))
Examiner notes Karakotsios is not limited to glasses. See [0031] “[i]n some embodiments a user might wear an earpiece or headset, rather than glasses, which can include at least a portion of this functionality. For example, the headset can include a microphone, speakers, and potentially haptic feedback elements, as well as communication elements, in order to enable the user to communicate with the computing device and receive information or notifications from the device. In at least some embodiments, the headset or earpiece might also include a camera that can be used to capture image information to determine a direction which the user's head is facing. The device might also, or alternatively, include one or more position and/or motion sensors for enabling a determination of the orientation of the device. Various other combinations or configurations can be used as well within the scope of the various embodiments.”
Examiner has incorporated Guo to address protentional deficiencies in Karakotsios. Guo teaches first and second earpieces (fig. 4 10s), worn on the ear and separate and each have at least one image device (fig. 2 (14s)).
[0061] In FIGS. 1 and 2 the housing 12 is adapted for being positioned at or behind a pinna 20, i.e. behind the ear of the user where it is held in place between the head and the pinna. In this position, an image capture device 14 is able to capture images in the direction of the field of view of the user as illustrated e.g. in FIG. 3. In FIGS. 1 and 2 the vertical part of the field of view is illustrated by the punctured lines emanating from the image capture device 14. This looking direction follow the direction of the nose of the user, and not the eyes. Guo
The test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
Therefore, Karakotsios teaches an ear piece with a camera to determine an object [0031] Guo teaches first and second earpieces (fig. 4 10s and fig. 2 14s) and newly cited reference Adsumilli bridges the gap of identify an object in an overlapping region of the first sensor data and the second sensor data.(fig. 6 (602-606)(fig. 5a (510)).
ConclusionAny inquiry concerning this communication or earlier communications from the examiner should be directed to GRANT SITTA whose telephone number is (571)270-1542. The examiner can normally be reached M-F 7:30-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patrick Edouard can be reached at 571-272-6084. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRANT SITTA/Primary Examiner, Art Unit 2622
Prosecution Insights