Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-4, 8, 9, 11-14, 18-21 are rejected under 35 U.S.C. 103 as being unpatentable over Anderson et al., United States Patent Publication 2015/0131913 (hereinafter “Anderson”), in view of Thomas et al., United States Patent Publication 20150084866 (hereinafter “Thomas”), in further view of Herz et al., United States Patent Publication 2009/0254971 (hereinafter “Herz”).
Claim 2:
Anderson discloses:
A mobile device comprising:
a display (see paragraph [0020]). Anderson teaches a display;
one or more processors coupled to the display (see paragraphs [0020], [0064] and [0065]). Anderson teaches a processor couples to a display ;
a camera configured to capture images of a region external to the display (see paragraphs [0018] and [0020]). Anderson teaches a camera to capture images of a region external to the display; and
a memory storing instructions that, when executed by the one or more processors (see paragraphs [0020], [0064] and [0065]). Anderson teaches a memory and processors, cause the one or more processors to perform operations comprising:
receiving image data from the camera, the image data including data representative of a surface acting as a virtual touchscreen in the region and a real object within the region (see paragraph [0046] and [0054]-[0056]). Anderson teaches receiving image data including a virtual touchscreen and a real object;
analyzing the image data to determine a three-dimensional (3D) orientation and location of the real object relative to the surface and a distance of the real object from the display (see paragraph [0054]). Anderson teaches analyzing the image data to recognize hands, fingers, finger gestures, hand movements and a location of hands relative to drawings;
producing an interaction event describing an interaction of the real object with the surface (see paragraph [0055]). Anderson teaches producing an interaction event to be performed based on the recognized gesture of the real object on the surface; and
Anderson fails to expressly disclose a replication of the real object superimposed on the display.
Thomas discloses:
causing the display to output display elements including (i) a first display element that includes a replication of the real object in the image superimposed on the display and oriented based, at least in part, on the 3D orientation and location of the real object relative to the surface and the distance from the display (see figure 1, paragraphs [0012] and [0013]). Thomas teaches the virtual hand may be translucently overlaid on a displayed element, such that the underlying displayed element is visible through the virtual hand, and
Anderson and Thomas fail to expressly disclose a representation of the history of the locations of the virtual object.
Herz discloses:
(ii) a second display element that includes a line representative of a history of a location of a point on the real object in the region (see paragraph [1185]). Herz teaches annotation the display to show the user previous interactions with the virtual object.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Anderson and Thomas to include displaying previous interactions with the virtual objects for the purpose of effectively displaying useful information for other users on a mobile phone, as taught by Herz.
Claim 3:
Anderson discloses:
wherein the point corresponds to a portion of the real object closest to the surface (see paragraph [0056]). Anderson teaches the touch gesture of a user's hand on the screen or surface is the corresponding point.
Claim 4:
Anderson discloses:
comparing the image to at least one other image to determine whether the location and the 3D orientation of the object corresponds to a gesture or a writing motion (see paragraph [0056]). Anderson teaches may include a touch screen sensor. Data from the sensor may be fed to hardware, software, firmware or a combination of the same to map/compare the touch gesture of a user's hand on the screen or surface to a corresponding drawing or image.
Claim 8:
Anderson fails to expressly disclose a representation of a second object.
Thomas discloses:
wherein the real object is a first hand or held by the first hand of a user and the image further includes a representation of a second object, wherein a combination of the 3D location and orientation of the real object and a position and orientation of the second object indicates a function to be performed on the display elements on the display (see paragraph [0017] and figure 2). Thomas teaches the hand includes a representation with a second object using the 3D location and orientation.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Anderson to include representing the second object with the hand for the purpose of effectively displaying interaction of 3D objects on a mobile phone, as taught by Thomas.
Claim 9:
Anderson discloses:
wherein the real object comprises a stylus held by the first hand of the user (see paragraph [0056]). Anderson teaches a pen stylus held by a first hand.
Claim 11:
Anderson fails to expressly discloses determining a touchdown event responsive to a distance.
Thomas discloses:
further comprising a software application, wherein the software application executes to determine the touchscreen event is a touchdown event responsive to the distance of at least a portion of the object above the surface being zero (see figures 1-7). Thomas discloses upon determining match against templates, system assigning attributes to the drawn element and providing the output to the application. Various applications are disclosed in paragraphs 24-28, 30-32 that support 3D touch interaction.) See Thomas fig. 4B and paragraph 43, A sense session may be initiated by entering an interaction space, entering a certain distance range from the input device and/or by touching the input device. The input device may detect that the sensed hand has entered the interaction space based on proximity detection (e.g., heat/infrared) and/or other detection indicating that the sensed hand is within a field of view or interaction space associated with the input device. The sense session may be associated with displaying the virtual hand on the display.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Anderson to include representing the second object with the hand for the purpose of effectively displaying interaction of 3D objects on a mobile phone, as taught by Thomas.
Claim 12:
A mobile device comprising:
a touchscreen display (see paragraph [0019]). Anderson teaches a touch display;
a depth camera directed to a space in front of the touchscreen display, the space including a surface acting as a virtual touchscreen (see paragraph [0019]-[0020]). Anderson teaches a depth camera in front of a touchscreen acting as a virtual space;
a processor configured to receive depth data from the depth camera (see paragraphs [0019]-[0020], [0064] and [0065]). Anderson teaches a memory and processors connected to a camera, the processor configured to:
determine, based on the depth data, a three-dimensional (3D) orientation of an object relative to the surface including a distance from the surface (see paragraph [0056]). Anderson teaches determining the orientation of an object relative to the surface;
produce a virtual touchscreen event describing an interaction of the object with the surface (see paragraph [0056]). Anderson teaches product an event based on the detected touch screen interaction;
provide the virtual touchscreen event to a software application capable of supporting 3D touch interactions (see paragraph [0056]). Anderson teaches data from the sensor may be fed to hardware, software, firmware or a combination of the same to map the touch gesture of a user's hand on the screen or surface to a corresponding drawing or image; and
Anderson fails to expressly disclose a replication of the real object superimposed on the display.
Thomas discloses:
cause the touchscreen display to output display elements including (i) a first display element that includes a replication of the object superimposed on the touchscreen display and oriented based, at least in part, on the 3D orientation and location of the object relative to the surface and the distance from the touchscreen display (see figure 1, paragraphs [0012] and [0013]). Thomas teaches the virtual hand may be translucently overlaid on a displayed element, such that the underlying displayed element is visible through the virtual hand, and
Anderson and Thomas fail to expressly disclose a representation of the history of the locations of the virtual object.
Herz discloses:
(ii) a second display element that includes a line representative of a history of a location of a point on the real object in the region (see paragraph [1185]). Herz teaches annotation the display to show the user previous interactions with the virtual object.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Anderson and Thomas to include displaying previous interactions with the virtual objects for the purpose of effectively displaying useful information for other users on a mobile phone, as taught by Herz.
Claims 13, 14, 18:
These claims are interpreted and rejected for the same reasons as the device of Claims 3, 4, and 8, respectively.
Claim 20-22:
Although these claims are method claims, they are interpreted and rejected for the same reasons as the device of Claims 1, 3, 4, respectively.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Anderson, in view of Thomas, in further view of Sagar et al., United States Patent Publication 20150084866 (hereinafter “Thomas”).
Claim 5:
Anderson discloses:
identifying, by the images, that one or more fingers are in contact with the surface (see figures 1-6). Anderson teaches identifying fingers in contact with the surface; and
Anderson and Thomas fail to expressly disclose identifying one or more fingers in contact for a predefined period of time.
Sagar discloses:
identifying, by the images, that one or more fingers are in contact with the surface for a specified period of time (see paragraph [0048]). Sagar teaches identifying interaction for the certain period of time; and
responsive to the identifying, driving the display to output a radial menu superimposed on the object (see paragraph [0048]). Sagar teaches identifying and driving a menu to be displayed.
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Anderson and Thomas to include an interaction was held for predetermined period of time for the purpose of effectively processing interactions of objections using gestures and touch, as taught by Sagar.
Claim 15:
This claim is interpreted and rejected for the same reasons as the device of Claim 5.
Claims 6, 7, 16 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Anderson, in view of Thomas, Herz and Sagar, in further view of Hinckley et al., United States Patent Publication 20110209088 (hereinafter “Thomas”).
Claim 6:
Sagar discloses:
identifying, by the images, that one or more fingers are in contact with the surface for a specified period of time (see paragraph [0048]). Sagar teaches identifying interaction for the certain period of time; and
Anderson, Thomas, Herz and Sagar fail to expressly disclose a radial menu and identifying fingers in contact.
Hinckley discloses:
wherein the radial menu is one of a plurality of radial menus and each of the plurality of radial menus is displayed responsive to identifying a different number of fingers are in contact with the surface (see paragraphs [0049] and [0105]). Hinckley teaches distinguishing which fingers are interacting with the radial menus on the surface; and
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Anderson, Thomas, Herz and Sagar to include a determination of the fingers in contact an interaction was held for predetermined period of time for the purpose of effectively precisely processing interactions of radial menus, as taught by Hinckley.
Claim 7:
Sagar discloses:
identifying, by the images, that one or more fingers are in contact with the surface for a specified period of time (see paragraph [0048]). Sagar teaches identifying interaction for the certain period of time; and
Anderson, Thomas, Herz and Sagar fail to expressly disclose a radial menu and identifying fingers in contact.
Hinckley discloses:
wherein the radial menus indicate directions to move the object to select a corresponding function of the radial menu and the operations further comprise:
identifying, based on the images, the direction of the object and performing the function corresponding to the direction (see paragraphs [0073] and [0081]). Hinckley teaches determining the direction of the fingers and performing the functions related to the direction; and
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Anderson, Thomas, Herz and Sagar to include determining the direction of the fingers for the menu for the purpose of effectively precisely processing interactions of radial menus, as taught by Hinckley.
Claims 16, 17:
These claims are interpreted and rejected for the same reasons as the device of Claims 6, 8, respectively.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Anderson, in view of Thomas and Herz, in further view of Al-Sharif et al., United States Patent Publication 20130321356 (hereinafter “Al-Sharif”).
Claim 10:
Anderson, Thomas and Herz fail to expressly disclose determining location and orientation corresponding to a writing motion if distance above surface is zero.
Al-Sharif discloses:
determining that the location and the 3D orientation of the object corresponds to a writing motion if a distance of at least a portion of the object above the surface is zero (see paragraphs [0026]). Al-Sharif teaches tracking data and determining what the user is writing and if the distance is below zero; and
Accordingly, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method disclosed by Anderson, Thomas and Herz to include determining if the writing distance is below a threshold of zero for the purpose of effectively determining the what the stylus is writing, as taught by Al-Sharif.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TIONNA M BURKE whose telephone number is (571)270-7259. The examiner can normally be reached M-F 8a-4p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen Hong can be reached at (571)272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TIONNA M BURKE/Examiner, Art Unit 2178 6/277/26
Prosecution Insights