DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 10-13, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Koker et al., US Patent Publication 2018/0299952.
Regarding independent claim 1, Koker et al. teaches a method for controlling an electronic device, wherein the electronic device comprises a display generation component (display controller described in paragraph 0031 to display on display device 618 of figure 6 as given in paragraph 0127) and one or more input devices (input devices 108 of figure 1 as given in paragraph 0031), and the input device comprises an eye tracking device (eye tracker sensor 612 of figure 6 as given in paragraphs 0127-0128 and 0156) and a gesture detection device (look down cameras 1108 and 1110 of figure 11 as given in paragraphs 0154-0155); and the method comprises:
displaying, through the display generation component, a physical environment of a user (via display device 618 of figure 6 as given in paragraph 0127);
generating, in response to a preset processing flow, at least one second object on a first object in the physical environment (paragraphs 0156-0157 and 0159 explain the generation of second virtual objects in the physical environment of the user), wherein the second object is a virtual object for calibrating the first object (paragraph 0159 explains the virtual object that is laid over the physical environment like a virtual keyboard or mouse with separate images for the left and right eye that is done to calibrate the view and gaze sensing as given in paragraph 0134, rendering the concept of the specific virtual object to calibrate another one to be obvious in view of the combination of teachings);
detecting, through the eye tracking device, that a gaze of the user points to a target second object of the second objects, to generate a visual indication object (paragraphs 0141-0142 explains how enhanced indications may be rendered based on the gaze of the user being on an object);
detecting a hand gesture of the user through the gesture detection device, and controlling the target second object based on the hand gesture and the visual indication object (paragraph 0155 explains how the detected gestures are used to render or control the virtual representation including the object); and
determining attribute information of the target second object in a case that a demand is satisfied by adjusting, based on the hand gesture, a fitting degree between the target second object and the first object (paragraphs 0128-0133 explain how the field of view and gaze and manipulation of the objects are used to determine the information of the object based on the fitting angle of the target viewed being within a certain field of view of the object).
Although all of the limitations are taught by Koker et al., they are spread out over various embodiments. It would have been obvious to one of ordinary skill in the art before the effective filing date to combine the embodiments as discussed above. The rationale to combine would be that various modifications and changes may be made thereto without departing from the broader spirit and scope of the embodiments as set forth by the invention (paragraph 0311 of Koker et al.).
Regarding claim 2, Koker et al. teaches the method according to claim 1, wherein detecting, through the eye tracking device, that the gaze of the user points to the target second object of the second objects comprises:
in a case of a glance or a gaze of a line of sight of the user at a second object, changing a score of the second object under the glance or the gaze of the line of sight or changing a score of a rest second object other than the second object under the glance or the gaze of the line of sight; and determining the target second object according to the score of the second object, wherein the second objects have corresponding scores (paragraph 0300 explains how the objects are given to be within the gaze of the user or outside of the gaze such that a score is seen or unseen to determine what the user is looking at to calibrate the object); or
taking as the target second object, in response to receiving first voice information indicating a selection intention, a second object to which a line of sight of the user points when the first voice information is received (although not discussed for selection, it is noted that paragraph 0127 explains the use of microphones to capture the environment of the user).
Regarding claim 3, Koker et al. teaches the method according to claim 1,
wherein the second object is a virtual object (paragraphs 0156-0157 and 0159 explain the generation of second virtual objects in the physical environment of the user); and/or
the attribute information of the target second object comprises one or more of a size of the target second object, a shape of the target second object, a position of the target second object, a posture of the target second object, and semantics of the target second object (paragraphs 0134 and 281 explain how orientations or positions of objects are determined by the gaze as part of the information); and/or
the fitting degree between the target second object and the first object is determined to satisfy the demand in response to a first type of gesture operation (paragraphs 0128-0133 explain how the field of view and gaze and gesture of manipulation of the objects are used to determine the information of the object based on the fitting angle of the target viewed being within a certain field of view of the object).
Regarding claim 4, Koker et al. teaches the method according to claim 1, wherein detecting the hand gesture of the user through the gesture detection device and controlling the target second object based on the hand gesture and the visual indication object comprise:
detecting the hand gesture of the user through the gesture detection device, and adjusting the attribute information of the target second object in combination with the visual indication object and in response to an adjustment instruction corresponding to the hand gesture (paragraphs 0156-0159 describe manipulation of virtual objects where paragraphs 0128-0133 explain how orientation or attribute of the object is detected based on the visual indication of the gaze of the user and the gesture of the manipulation).
Regarding independent claim 10, Koker et al. teaches an electronic device, comprising:
at least one memory and at least one processor (as given in paragraphs 0031 and 0036);
wherein the at least one memory is configured to store program codes, and the program codes, when called by the at least one processor (as given by the codes and instructions described in paragraph 0274), cause the at least one processor to:
display, through a display generation component, a physical environment of a user (via display device 618 of figure 6 as given in paragraph 0127);
generate, in response to a preset processing flow, at least one second object on a first object in the physical environment (paragraphs 0156-0157 and 0159 explain the generation of second virtual objects in the physical environment of the user), wherein the second object is a virtual object for calibrating the first object (paragraph 0159 explains the virtual object that is laid over the physical environment like a virtual keyboard or mouse with separate images for the left and right eye that is done to calibrate the view and gaze sensing as given in paragraph 0134, rendering the concept of the specific virtual object to calibrate another one to be obvious in view of the combination of teachings);
detect, through an eye tracking device, that a gaze of the user points to a target second object of the second objects, to generate a visual indication object (paragraphs 0141-0142 explains how enhanced indications may be rendered based on the gaze of the user being on an object);
detect a hand gesture of the user through a gesture detection device, and control the target second object based on the hand gesture and the visual indication object (paragraph 0155 explains how the detected gestures are used to render or control the virtual representation including the object); and
determine attribute information of the target second object in a case that a demand of the user is satisfied by adjusting, based on the hand gesture, a fitting degree between the target second object and the first object (paragraphs 0128-0133 explain how the field of view and gaze and manipulation of the objects are used to determine the information of the object based on the fitting angle of the target viewed being within a certain field of view of the object).
Although all of the limitations are taught by Koker et al., they are spread out over various embodiments. It would have been obvious to one of ordinary skill in the art before the effective filing date to combine the embodiments as discussed above. The rationale to combine would be that various modifications and changes may be made thereto without departing from the broader spirit and scope of the embodiments as set forth by the invention (paragraph 0311 of Koker et al.).
Regarding claim 11, Koker et al. teaches the electronic device according to claim 10, wherein the at least one processor, when caused to detect, through the eye tracking device, that the gaze of the user points to the target second object of the second objects, is to:
in a case of a glance or a gaze of a line of sight of the user at a second object, change a score of the second object under the glance or the gaze of the line of sight or change a score of a rest second object other than the second object under the glance or the gaze of the line of sight; and determine the target second object according to the score of the second object, wherein the second objects have corresponding scores (paragraph 0300 explains how the objects are given to be within the gaze of the user or outside of the gaze such that a score is seen or unseen to determine what the user is looking at to calibrate the object); or
take as the target second object, in response to receiving first voice information indicating a selection intention, a second object to which a line of sight of the user points when the first voice information is received (although not discussed for selection, it is noted that paragraph 0127 explains the use of microphones to capture the environment of the user).
Regarding claim 12, Koker et al. teaches the electronic device according to claim 10, wherein
the second object is a virtual object (paragraphs 0156-0157 and 0159 explain the generation of second virtual objects in the physical environment of the user); and/or
the attribute information of the target second object comprises one or more of a size of the target second object, a shape of the target second object, a position of the target second object, a posture of the target second object, and semantics of the target second object (paragraphs 0134 and 281 explain how orientations or positions of objects are determined by the gaze as part of the information); and/or
the fitting degree between the target second object and the first object is determined to satisfy the demand in response to a first type of gesture operation (paragraphs 0128-0133 explain how the field of view and gaze and gesture of manipulation of the objects are used to determine the information of the object based on the fitting angle of the target viewed being within a certain field of view of the object).
Regarding claim 13, Koker et al. teaches the electronic device according to claim 10, wherein the at least one processor, when caused to detect the hand gesture of the user through the gesture detection device and control the target second object based on the hand gesture and the visual indication object, is to:
detect the hand gesture of the user through the gesture detection device, and adjust the attribute information of the target second object in combination with the visual indication object and in response to an adjustment instruction corresponding to the hand gesture (paragraphs 0156-0159 describe manipulation of virtual objects where paragraphs 0128-0133 explain how orientation or attribute of the object is detected based on the visual indication of the gaze of the user and the gesture of the manipulation).
Regarding independent claim 19, Koker et al. teaches a non-transitory computer-readable storage medium, configured to store program codes, wherein the program codes, when run by a processor (as given in paragraph 0132 and 0136), cause the processor to:
display, through a display generation component, a physical environment of a user (via display device 618 of figure 6 as given in paragraph 0127);
generate, in response to a preset processing flow, at least one second object on a first object in the physical environment (paragraphs 0156-0157 and 0159 explain the generation of second virtual objects in the physical environment of the user), wherein the second object is a virtual object for calibrating the first object (paragraph 0159 explains the virtual object that is laid over the physical environment like a virtual keyboard or mouse with separate images for the left and right eye that is done to calibrate the view and gaze sensing as given in paragraph 0134, rendering the concept of the specific virtual object to calibrate another one to be obvious in view of the combination of teachings);
detect, through an eye tracking device, that a gaze of the user points to a target second object of the second objects, to generate a visual indication object (paragraphs 0141-0142 explains how enhanced indications may be rendered based on the gaze of the user being on an object);
detect a hand gesture of the user through a gesture detection device, and control the target second object based on the hand gesture and the visual indication object (paragraph 0155 explains how the detected gestures are used to render or control the virtual representation including the object); and
determine attribute information of the target second object in a case that a demand of the user is satisfied by adjusting, based on the hand gesture, a fitting degree between the target second object and the first object (paragraphs 0128-0133 explain how the field of view and gaze and manipulation of the objects are used to determine the information of the object based on the fitting angle of the target viewed being within a certain field of view of the object).
Although all of the limitations are taught by Koker et al., they are spread out over various embodiments. It would have been obvious to one of ordinary skill in the art before the effective filing date to combine the embodiments as discussed above. The rationale to combine would be that various modifications and changes may be made thereto without departing from the broader spirit and scope of the embodiments as set forth by the invention (paragraph 0311 of Koker et al.).
Regarding claim 20, Koker et al. teaches the non-transitory computer-readable storage medium according to claim 19, wherein
the second object is a virtual object (paragraphs 0156-0157 and 0159 explain the generation of second virtual objects in the physical environment of the user); and/or
the attribute information of the target second object comprises one or more of a size of the target second object, a shape of the target second object, a position of the target second object, a posture of the target second object, and semantics of the target second object (paragraphs 0134 and 281 explain how orientations or positions of objects are determined by the gaze as part of the information); and/or
the fitting degree between the target second object and the first object is determined to satisfy the demand in response to a first type of gesture operation (paragraphs 0128-0133 explain how the field of view and gaze and gesture of manipulation of the objects are used to determine the information of the object based on the fitting angle of the target viewed being within a certain field of view of the object).
Allowable Subject Matter
Claims 5-9 and 14-18 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: none of the prior art of record teach the specific combination of limitations, especially “the second object is a virtual plane indication object corresponding to a floor in a case that the first object is the floor; and … detect[ing] the hand gesture of the user through the gesture detection device and control[ling] the target second object based on the hand gesture and the visual indication object…: lift[ing] or lower[ing] the plane indication object in response to determining that the visual indication object is located at the plane indication object and the hand gesture is a second type of gesture operation” as recited in claims 5 and 14 and “wherein the second object is a skirting line model of each skirting line of a wall surface in a case that the first object is the wall surface…[calibrate/calibrating] the target skirting line model in response to determining that the visual indication object is located on the target skirting line model and the hand gesture is a third type of gesture operation; and … [determine/determining] attribute information of the target second object in a case that the demand is satisfied by adjusting, based on the hand gesture, the fitting degree between the target second object and the first object…: [determine/determining] attribute information of a calibrated skirting line model after the calibrated skirting line model forms a virtual closed surface, and [take/taking] the virtual closed surface as a virtual wall surface model for calibrating the wall surface” as recited in claims 6 and 15 and “wherein the second object is a virtual shape model corresponding to a piece of furniture in a case that the first object is the piece of furniture…detect[ing], through the eye tracking device, that the gaze of the user points to the target second object of the second objects…: [determine/determining] a target shape model to which a line of sight of the user points as the target second object according to the line of sight; and … [determine/determining] attribute information of the target second object in a case that the demand is satisfied by adjusting, based on the hand gesture, a fitting degree between the target second object and the first object…: [determine/determining] attribute information of the target shape model in response to a fourth type of gesture operation” as recited in claims 7 and 16. All other claims are objected to as being dependent on an allowable base claim.
Response to Arguments
Applicant's arguments filed 8/5/26 have been fully considered but they are not persuasive.
Applicant contends that Koker does not teach the same calibration virtual object on the physical object, rendering the claims to be allowable. The examiner disagrees. These features were not previously recited and have now been rejected appropriately above. Paragraph 0159 of Koker explains the virtual object that is laid over the physical environment like a virtual keyboard or mouse with separate images for the left and right eye that is done to calibrate the view and gaze sensing as given in paragraph 0134, rendering the concept of the specific virtual object to calibrate another one to be obvious in view of the combination of teachings.
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., the second object is a virtual object for calibrating the first object) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Applicant contends that Koker does not teach the same determination of the attribute information based on the given fitting degree, rendering the claims to be allowable. The examiner disagrees. While Koker may not use the same specific terms as the instant application, there is still a teaching of matching and calibration, especially in paragraph 0133 based on the manipulation of paragraph 0131. The phrase “attribute information” has been interpreted broadly in this situation to encompass the given information.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The closest prior art is made of record in the attached notice of references cited.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PARUL H GUPTA whose telephone number is (571)272-5260. The examiner can normally be reached Monday through Friday, from 10 AM to 7 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ke Xiao can be reached at 571-272-7776. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PARUL H GUPTA/Primary Examiner, Art Unit 2627
Prosecution Insights