DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 7/30/2026 have been fully considered but they are not persuasive.
Applicant contends:
PNG
media_image1.png
208
716
media_image1.png
Greyscale
PNG
media_image2.png
74
684
media_image2.png
Greyscale
PNG
media_image3.png
122
692
media_image3.png
Greyscale
Examiner respectfully disagrees.
MPEP 2111 states, During patent examination, the pending claims must be “given their broadest reasonable interpretation consistent with the specification.” The Federal Circuit’s en banc decision in Phillips v. AWH Corp., 415 F.3d 1303, 1316, 75 USPQ2d 1321, 1329 (Fed. Cir. 2005) expressly recognized that the USPTO employs the “broadest reasonable interpretation” standard: The Patent and Trademark Office (“PTO”) determines the scope of claims in patent applications not solely on the basis of the claim language, but upon giving claims their broadest reasonable construction “in light of the specification as it would be interpreted by one of ordinary skill in the art.” In re Am. Acad. of Sci. Tech. Ctr., 367 F.3d 1359, 1364[, 70 USPQ2d 1827, 1830] (Fed. Cir. 2004). Indeed, the rules of the PTO require that application claims must “conform to the invention as set forth in the remainder of the specification and the terms and phrases used in the claims must find clear support or antecedent basis in the description so that the meaning of the terms in the claims may be ascertainable by reference to the description.” 37 CFR 1.75(d)(1).
Examiner believes Applicant is narrowly viewing the “identifying” as recognizing a particular object, as opposed to recognizing its existence. Adsumilli identifies an object by noting the existence in two images and features such as depth from the background and then uses feature detection to identify the object in order to stitch the images. Adsumilli states:
[0024] A second example of an image stitching algorithm that is a higher quality image stitching algorithm than the first example image stitching algorithm can analyze the depth of image features within an overlap region of two or more images. For instance, for an object (such as a vehicle, person, or tree) within a common field of view for two cameras, the depth of the object can be identified and associated with the image feature within each image captured by the two cameras corresponding to the object. Image feature depth can be determined in any suitable way, for instance based on parallax information, based on a known size of the corresponding object and the dimensions of the image feature within the image, and the like.
[0039] The feature detection module 250 is configured to identify and classify image features within images received or stored by the image server 205. In some embodiments, the feature detection module can detect humans, human faces, human hands or limbs, vehicles, animals, plants or trees, edges or surfaces of objects, lines or curves within the image, resolutions or textures within the image, background objects, or any other suitable image feature present within an image. The feature detection module 250 can classify the priority of each detected feature, for instance as “high priority” or “low priority”. In some embodiments, the priority classification assigned by the feature detection module 250 to an image feature corresponds to the importance of the image feature, the likelihood that a low quality stitching algorithm will produce undesirable image artifacts within the image feature, or the likelihood that a user viewing a stitched image will notice a distortion of the image feature resulting from the stitching. For example, human faces and image edges can be classified as “high priority” (thus requiring a high quality stitching operation) while background textures and objects (such as leafy plants or the sky) can be classified as “low priority” (in which cases, a low quality stitching operation may be suitable).
PNG
media_image4.png
788
730
media_image4.png
Greyscale
Applicant further contends:
PNG
media_image5.png
334
722
media_image5.png
Greyscale
In response to applicant's argument, Examiner notes, the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
Karakotsios teaches an electronic device can attempt to determine a gaze direction of a user, which can be used to determine an object of interest to the user. Determining the gaze direction helps to reduce the search space and reduce processing requirements for identifying the object. Image information can be captured that includes the object of interest, which then can be analyzed to recognize the type of object. Upon recognizing the object, the user can be provided with information about the object, which in some cases can depend at least in part upon a current context or location of the object. If the object is a networked device, the user can potentially be provided with input options for controlling the device.(abstract).
Karakotsios goes on to state: [0024] In this example where the device has determined the object of interest, the device can attempt to recognize the object as discussed elsewhere herein. Upon determining a type of the object 206, the device can determine the type of interactions that the user can have with respect to that object. For example, if the object is a book that the user does not own, there might be information available that the user can obtain about the book, such as author and title, reviews, versions available, and the like. If, on the other hand, the object is an electronic device with which the computing device 204 is able to communicate, such as over at least one network, the computing device can determine which inputs or controls are available for that type of electronic device, and can provide at least a portion of those inputs or controls to the user. For example, if the electronic device is an Internet radio the computing device might enable the user to use the touch screen or other portions of the computing device to adjust a volume of the radio, change the station, turn off the device, or perform another such action. In some embodiments, a user gazing at a device to select the device can provide an audible command, perform a gesture, provide a motion, or indicate another such input that can be used to control an aspect of the device. For example, a user can say “on” to activate the device, might make an “up” motion with a hand to increase the volume, or might squeeze a pressure-sensitive portion of the device to change to the next “favorite” station, among other such actions.
Karakotsios and Guo fail to teach identifying an object in an overlapping region of the first sensor data and the second sensor data.
However, as noted above Adsumilli teaches identifying an object in an overlapping region of the first sensor data and the second sensor data.(fig. 1a (105a and 105b )(fig. 6 (602-606)(fig. 5a (510)).
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios and Guo to further include identifying an object in an overlapping region of the first sensor data and the second sensor data as taught by Adsumilli in order to aid in the selection and calibration of multi-camera scenario [0036]
Therefore, Examiner believes it is well within the purview of one of ordinary skill in the art to modify the identifying (abstract) and gesture [0024] of Karakotsios to include multiple images and “identifying” of Adsumilli in order to aid in the selection and calibration of multi-camera environment. As multi-cameras can add depth, wider field of view and more complex image identification (i.e., using different types of cameras such as IR and depth cameras).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5, 7-11, 13, and 17-20 is/are rejected under 35 U.S.C. 103(a) as being obvious over Karakotsios (2016/0283778) hereinafter, Karakotsios in view of Guo et al (2016/0183014), further in view of Adsumilli et al (2017/0006220) hereinafter, Adsumilli.
In regards to claim 1, a Karakotsios teaches method comprising (abstract):
obtaining first sensor data from a first device worn on a head (fig. 3a (322) [0027);
obtaining second sensor data from a second device worn on the head, wherein the first sensor data and the second sensor data correspond to a first time [0027-032]; ;[0018, 0021, 0030-0032] (fig. 2 (202) and 206)) and
[0027] As illustrated in FIG. 3(b), there can be at least one camera 322 on the front of the glasses as well. This camera can be used to capture image information about objects that might be within the user's line of sight. The image information can be captured and analyzed to determine the relative direction of one or more objects, which can be used with the gaze direction information to determine the object at which the user is gazing. In some embodiments, stereoscopic cameras or multiple cameras can be used to determine three-dimensional information about the relative positions of the objects with respect to the glasses, but in general the proximity of the glasses to the user's eyes means that the point of view will be substantially similar, such that a direction to an object from a user's eyes will be almost the same as the direction from the glasses to that object. Thus, processing needs can be reduced while using the glasses. Further, as discussed, such a device enables the gaze to be determined even when the user is turning his or her head, or might otherwise not be facing a computing device.
[0030] In some embodiments where high resolution and/or precision might not be needed, the glasses might utilize only one or more front-facing cameras in order to identify an object, or at least narrow the search field. For example, a user might turn his or her head to the left or the right, which can give a general indication of the direction in which the user is looking. The user also can be trained to face an object of interest in such a mode if the viewing direction is to be interpreted as input, instead of just moving the user's eyes while keeping the head substantially stationary. To save power in some embodiments, a pair of glasses might first attempt to determine an object of interest based on the direction the glasses are pointing, and then activate the gaze determination components if the object cannot be determined with at least an acceptable amount of certainty. In embodiments where the direction of the glasses is used to identify objects, information from sensors in the glasses, such as position sensors (e.g., GPS), motion sensors (e.g., gyroscopes, accelerometers, etc.), and the like, can be used to determine the relative position, orientation, and/or direction of the glasses. Various other approaches can be used as well within the scope of the various embodiments.
PNG
media_image6.png
672
448
media_image6.png
Greyscale
PNG
media_image7.png
678
440
media_image7.png
Greyscale
in response to receiving a selection input, performing a selection of the object [023-0025, 0032-0033,0046] “In some embodiments, a user gazing at a device to select the device can provide an audible command, perform a gesture, provide a motion, or indicate another such input that can be used to control an aspect of the device. For example, a user can say “on” to activate the device, might make an “up” motion with a hand to increase the volume, or might squeeze a pressure-sensitive portion of the device to change to the next “favorite” station, among other such actions.”[0024] (fig. 4 402-412))
PNG
media_image8.png
730
582
media_image8.png
Greyscale
Karakotsios fails to expressly teach a first earn worn device worn by a user; a second ear worn device worn by the user.
However, Guo teaches a first earn worn device worn by a user; a second ear worn device worn by the user.[0021, 0026, 0031, 0061-0076],(fig. 2 and 4 (14 imaging device and fig. 5 (detecting an object)) Guo
1
PNG
media_image9.png
558
702
media_image9.png
Greyscale
[0021] The image capture device may be positioned in a housing of the hearing device, and the hearing device is configured to be positioned at an ear of the user. This allows detection of e.g. movement of the head and/or capturing images from a well-defined direction relative to the head of the user. The image capture device may alternatively be positioned remote from a housing of the hearing device and the image capture device may then be in wired or wireless communication with the processor. Memory devices may be included for buffering and/or storing images. A remote location may be needed if images are to be captured in the direction of the face of the user and the hearing device is positioned at least partly in the ear canal of the user. This may e.g. be achieved via an image capture device mounted in or on a set of glasses or in a body-worn housing, e.g. for being worn at the chest of the user or a device located on the pinna and either in wired or wireless communication with the in-ear part. Multiple image capture devices may be combined, e.g. to ensure a more complete field of view. Also, image capture devices facing different directions. As an example of multiple image capture device may be one or more looking forward and one or more looking backwards.
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios to further include a first earn worn device worn by a user; a second ear worn device worn by the user as taught by Guo in order to allow for a smaller packaged device that doesn’t obscure the view of the user. Examiner notes Guo, like Karakotsios, teaches using either a glasses form or ear buds [0031].
Karakotsios and Guo fail to teach identifying an object in an overlapping region of the first sensor data and the second sensor data.
However, Adsumilli teaches identifying an object in an overlapping region of the first sensor data and the second sensor data.(fig. 1a (105a and 105b )(fig. 6 (602-606)(fig. 5a (510))
PNG
media_image10.png
774
756
media_image10.png
Greyscale
PNG
media_image11.png
830
632
media_image11.png
Greyscale
PNG
media_image12.png
594
558
media_image12.png
Greyscale
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios and Guo to further include identifying an object in an overlapping region of the first sensor data and the second sensor data as taught by Adsumilli in order to aid in the selection and calibration of multi-camera scenario [0036]
Therefore, Karakotsios in view of Guo and Adsumulli teaches in response to receiving a selection input, performing a selection of the object [023-0025, 0032-0033,0046] “In some embodiments, a user gazing at a device to select the device can provide an audible command, perform a gesture, provide a motion, or indicate another such input that can be used to control an aspect of the device. For example, a user can say “on” to activate the device, might make an “up” motion with a hand to increase the volume, or might squeeze a pressure-sensitive portion of the device to change to the next “favorite” station, among other such actions.”[0024] (fig. 4 402-412)) Karakotsios identified in the overlapping region. (fig. 1a (105a and 105b )(fig. 6 (602-606)(fig. 5a (510)) Adsumulli.
In regards to claim 9, Karakotsios teaches a non-transitory computer readable medium comprising computer readable code executable by one or more processor to (fig. 8 (802-812)), : obtain first sensor data from a first device worn on a head (fig. 3a (322) [0027-0032]);; obtain second sensor data from a second device worn on the head, wherein the first sensor data and the second sensor data correspond to a first time; [0027-032]; [0018, 0021, 0030] (fig. 2 (202) and 206)); and in response to receiving a selection input, perform selection operation on the object [0024-0025,0033, 0046] (fig. 4 402-412))
Karakotsios fails to expressly teach a first earn worn device worn by a user; a second ear worn device worn by the user.
However, Guo teaches a first earn worn device worn by a user; a second ear worn device worn by the user.[0021, 0026, 0031, 0061-0076],(fig. 2 and 4 (14 imaging device and fig. 5 (detecting an object))fig. 8 (14)).
PNG
media_image13.png
582
836
media_image13.png
Greyscale
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios to further include a first earn worn device worn by a user; a second ear worn device worn by the user as taught by Guo in order to allow for a smaller packaged device that doesn’t obscure the view of the user. Examiner notes Guo teaches using a glass form or ear buds [0031].
Karakotsios and Guo fail to teach identify an object in an overlapping region of the first sensor data and the second sensor data.
However, Adsumilli teaches identify an object in an overlapping region of the first sensor data and the second sensor data.(fig. 6 (602-606)(fig. 5a (510))
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios and Guo to further include identifying an object in an overlapping region of the first sensor data and the second sensor data as taught by Adsumilli in order to aid in the selection and calibration of multi-camera scenario [0036]
Therefore, Karakotsios in view of Guo and Adsumulli teaches in response to receiving a selection input, performing a selection of the object [023-0025, 0032-0033,0046] “In some embodiments, a user gazing at a device to select the device can provide an audible command, perform a gesture, provide a motion, or indicate another such input that can be used to control an aspect of the device. For example, a user can say “on” to activate the device, might make an “up” motion with a hand to increase the volume, or might squeeze a pressure-sensitive portion of the device to change to the next “favorite” station, among other such actions.”[0024] (fig. 4 402-412)) Karakotsios identified in the overlapping region. (fig. 1a (105a and 105b )(fig. 6 (602-606)(fig. 5a (510)) Adsumulli.
In regards to claim 17, Karakotsios teaches comprising: one or more processors; and one or more computer readable media comprising computer readable code executable by the one or more processors to (fig. 8 (802-812)): obtain first sensor data from a first device worn on a head (fig. 3a (322) [0027-0032);; obtain second sensor data from a second device worn on the head, wherein the first sensor data and the second sensor data correspond to a first time[0027-032]; determine a head position for the head based on the first sensor data and the second sensor data; ;[0018, 0021, 0030] (fig. 2 (202) and 206)) and in response to receiving a selection input, perform a selection operation on the object. [0024-0025,0033 0046] ] (fig. 4 402-412))
Karakotsios fails to expressly teach a first earn worn device worn by a user; a second ear worn device worn by the user.
However, Guo teaches a first earn worn device worn by a user; a second ear worn device worn by the user.[0021, 0026, 0031, 0061-0076],(fig. 2 and 4 (14 imaging device and fig. 5 (detecting an object))
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios to further include a first earn worn device worn by a user; a second ear worn device worn by the user as taught by Guo in order to allow for a smaller packaged device that doesn’t obscure the view of the user. Examiner notes Guo teaches using a glass form or ear buds [0031].
Karakotsios and Guo fail to teach identify an object in an overlapping region of the first sensor data and the second sensor data.
However, Adsumilli teaches identify an object in an overlapping region of the first sensor data and the second sensor data.(fig. 6 (602-606)(fig. 5a (510))
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios and Guo to further include identify an object in an overlapping region of the first sensor data and the second sensor data as taught by Adsumilli in order to aid in the selection and calibration of multi-camera scenario [0036].
Therefore, Karakotsios in view of Guo and Adsumulli teaches in response to receiving a selection input, performing a selection of the object [023-0025, 0032-0033,0046] “In some embodiments, a user gazing at a device to select the device can provide an audible command, perform a gesture, provide a motion, or indicate another such input that can be used to control an aspect of the device. For example, a user can say “on” to activate the device, might make an “up” motion with a hand to increase the volume, or might squeeze a pressure-sensitive portion of the device to change to the next “favorite” station, among other such actions.”[0024] (fig. 4 402-412)) Karakotsios identified in the overlapping region. (fig. 1a (105a and 105b )(fig. 6 (602-606)(fig. 5a (510)) Adsumulli.
In regards to claim 2, Karakotsios in view of Guo and Adsumilli teaches method of claim 1, wherein the first sensor data comprises first orientation data, and wherein the second sensor data comprises second orientation data [0030-0032]. Karakotsios and [0032] Guo
In regards to claim 3, Karakotsios in view of Guo and Adsumilli teaches method of claim 1, wherein the first sensor data comprises first image data, wherein the second sensor data further comprises second image data, and determinea direction of the head based on an overlap between the first image data and the second image data [0027-0032] stereoscopic cameras must have overlap. Karakotsios
In regards to claim 5, Karakotsios in view of Guo and Adsumilli teaches method of claim 1, wherein the selection input comprises a selection gesture [0024,0033, 0043]. Karakotsios
In regards to claim 7, Karakotsios in view of Guo and Adsumilli teaches the method of claim 1, wherein the selection input is detected on an accessory device, and wherein the selection input is received from the accessory device [0033,0043, 0051, 0062] touch on screen. Karakotsios
In regards to claim 8, Karakotsios in view of Guo and Adsumilli teaches wherein the first device comprises a first earbud [0031] Karakotsios and wherein the second device comprises a second earbud of the earbud pair (fig. 2 and fig. 4 14 )Guo).
In regards to claim 10, Karakotsios in view of Guo and Adsumilli teaches non-transitory computer readable medium of claim 9, wherein the first sensor data comprises first orientation data, and wherein the second sensor data comprises second orientation data [0027,0031,0032,0042,0049,0052] . Karakotsios and [0021, 0026, 0031-0032, 0061-0076] Guo.
In regards to claim 11, Karakotsios in view of Guo and Adsumilli teaches non-transitory computer readable medium of claim 9, wherein the first sensor data comprises first image data, further comprising computer readable code wherein the second sensor data comprises second image data, and determine a direction of the head based on an overlap between the first image data and the second image data. [0027-0032] stereoscopic cameras must have overlap. Karakotsios
In regards to claim 13, Karakotsios in view of Guo and Adsumilli teaches non-transitory computer readable medium of claim 9, wherein the selection input comprises a selection gesture. [0024,0033, 0043]. Karakotsios
In regards to claim 18, Karakotsios in view of Guo and Adsumilli teaches system of claim 17, wherein the one or more processors and the one or more computer readable media are comprised in a mobile device communicably coupled to an audio headset comprising the first device and the second device [0031]. Karakotsios
In regards to claim 19, Karakotsios in view of Guo and Adsumilli teaches system of claim 17, wherein the first sensor data comprises first orientation data, and wherein the second sensor data comprises second orientation data. [0027,0031,0032,0042,0049,0052] . Karakotsios
In regards to claim 20, Karakotsios in view of Guo and Adsumilli teaches system of claim 17, wherein the first sensor data comprises first image data, wherein the second sensor data comprises second image data, comprising computer readable code to and determine a direction of the head based on an overlap between the first image data and the second image data. [0027-0032] stereoscopic cameras must have overlap. Karakotsios
Claim(s) 6 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Karakotsios and Guo and Adsumilli in view of Mekenkamp (2009/0251559) hereinafter, Mekenkamp.
In regards to claim 6, Karakotsios and Guo and Adsumilli fails to teach the method of claim 1, further comprising: generating audio feedback corresponding to identifying the object. Examiner notes Guo appears to provide haptic feedback [0035]
However, Mekenkamp teaches further comprising: generating audio feedback corresponding to identifying the object.[0008].
It would have been obvious to one of ordinary skill in the art to modify the teachings of Karakotsios and Guo to further include teaches further comprising: generating audio feedback regarding the object in response to identifying the object to provide feedback to the user when the object is usable [0008].
In regards to claim 14, Karakotsios and Guo and Adsumilli in view of Mekenkamp teaches the non-transitory computer readable medium of claim 9,see rational of claim 6, further comprising computer readable code to: generate audio feedback corresponding in response to identifying the object [0008] Mekenkamp.
Allowable Subject Matter
Claims 15-16 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GRANT SITTA whose telephone number is (571)270-1542. The examiner can normally be reached M-F 7:30-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patrick Edouard can be reached at 571-272-6084. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRANT SITTA/Primary Examiner, Art Unit 2622
Prosecution Insights