DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements filed 04/30/2025 has been acknowledged and considered by the examiner. An initialed copy of the PTO-1449 is included in this correspondence.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 3-4, 6-8, 10-11, 13-15, 17-18, and 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US Pub. 2013/0113738 A1) in view of Cederlund et al. (US Pub. 2015/0130740 A1).
Regarding claim 1; Seok teaches an electronic device (a mobile terminal 120, Fig.1) comprising:
one or more sensors (a photograph unit 520 and a sensing unit 560, Fig.5);
communication circuitry (a communication unit 540, Fig.5);
one or more processors (a controller 510, Fig.5); and memory (a storage unit 570, Fig.5) storing instructions configured to be executed by the one or more processors (para. [0062]), the instructions for:
obtaining, via a first subset of the one or more sensors, an image, wherein the image includes a display (Fig.1, para. [0011, 0034, and 0057], the mobile terminal 120 is configured to display an image of a remote screen 111 as a camera screen. The controller is configured to define a remote screen area in the image of the captured remote screen, to perform screen mapping between the rectangle bounds of remote screen area and the real remote screen, to generate, if user input events related to the rectangle of the remote screen are occurred, event information for controlling an object corresponding to the user input signal based on the results of the screen mapping, and to transmit the event information to the remote apparatus through the communication unit);
obtaining, via a second subset of the one or more sensors (a user input unit 550, Fig.5), an input (Fig.6, para. [0038 and 0059], if the user may perform an input corresponding to a remote screen area on the mobile terminal 120; the controller may generate an event information for controlling at least one of the objects, based on the results of the screen mapping);
determining, using at least the input and the image, a location on the display corresponding to the input (para. [0038], the mobile terminal 120 detects a position of a user input event on the mobile terminal 120. Then, the mobile terminal 120 converts the detected position of the user input event into the corresponding position on the remote screen 111 using the conversion matrix acquired in operation 250. That is, the mobile terminal 120 may apply the detected position of the user input event to the conversion matrix to thereby calculate the corresponding coordinate values on the remote screen 111);
obtaining, via the second subset of the one or more sensors, a gesture input (para. [0007, 0010, and 0088-0093], the mobile terminal 120 comprises a multi-touch interface configured to detect a multi-touch gesture input); and
transmitting information associated with the location on the display and the gesture input to an external electronic device using the communication circuitry (para. [0012-0014, 0038-0039, and 0055-0056 ], the mobile terminal 120 generates event information associated with the location of the user input event to the remote apparatus 110 using the communication unit. It is noted that “event information” would be associated with the location on the remote screen and the gesture input).
Lee does not teach that the input includes a gaze input.
Cederlund teaches that the input includes the input includes a gaze input (Figs. 1 and 2, para. [0002, 0012, and 0041], Cederlund discloses a method for assisting a user when interacting with a graphical user interface by combining eye based input with gesture based input and gesture based user commands. In particular, a control module 36 receives gaze data from a gaze tracking module 40 and gesture data from a user input means 50 to generate a command to control graphical information presentation area on a display 20. For example, an object may be selected by gazing at the object and pressing down and holding a finger on a touchpad).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the method of Lee for controlling content on remote screen to include the method of Cederlund of controlling a user interface of a display by combining a gaze input and a gesture input. Accordingly, in the system of Lee as modified by Cederlund, event information associated with a gaze input and a gesture input would be transmitted to the remote screen. The motivation would have been in order to provide a user friendly and intuitive interaction with graphical user interfaces (Cederlund, para. [0012]).
Regarding claim 3; Lee in view of Cederlund teaches the electronic device of claim 1 as discussed above. Lee further teaches after obtaining the image (Fig.2, in an operation 240, the method comprises defining remote screen area after capturing an image of the remote screen and displaying the captured image on the mobile terminal 120), determining a location of the display relative to the electronic device using the image (para. [0041], the method comprises determining whether there is a change in motion of the mobile terminal 120. If so, the mobile terminal 120 may define a new remote screen area. In other words, a change in motion of the mobile terminal would be equivalent to a change in a location of the remote screen 111 with respect to the mobile terminal 120); and after determining the location of the display relative to the electronic device using the image, continually tracking the location of the display relative to the electronic device using data from a third subset of the one or more sensors (para. [0061], the sensing unit 560 detects a change in motion of the mobile terminal 120, including a change in location or position of the mobile terminal 120 with respect to the remote apparatus 110).
Regarding claim 4; Lee in view of Cederlund teaches the electronic device of claim 3 as discussed above. Lee further teaches the third subset of the one or more sensors comprises a motion sensor (para. [0061], the sensing unit 560 may be configured to include a gravity sensor, a gyro sensor, a geo-magnetic sensor, etc.).
Regarding claim 6; Lee in view of Cederlund teaches the electronic device of claim 1 as discussed above. Lee does not teach the gesture input comprises a hand gesture or a head gesture.
Cederlund teaches the gesture input comprises a hand gesture or a head gesture (para. [0028, 0090, 0179, 0215, and 0219], the gesture input comprises a hand gesture).
The motivation is the same as the rejection of claim 1.
Regarding claim 7; Lee in view of Cederlund teaches the electronic device of claim 1 as discussed above. Lee further teaches the external electronic device comprises a source device that provides images to a television that comprises the display (para. [0029], the remote apparatus 110 may include a smart TV) and wherein the electronic device is a cellular telephone or a tablet computer (para. [0029], the mobile terminal 120 may be a mobile phone).
Regarding claim 8; Lee in view of Cederlund teaches a method of operating an electronic device that comprises one or more sensors and communication circuitry, the method comprising: obtaining, via a first subset of the one or more sensors, an image, wherein the image includes a display; obtaining, via a second subset of the one or more sensors, a gaze input; determining, using at least the gaze input and the image, a location on the display corresponding to the gaze input; obtaining, via the second subset of the one or more sensors, a gesture input; and transmitting information associated with the location on the display and the gesture input to an external electronic device using the communication circuitry (similar to the analysis of claim 1 above).
Regarding claim 10; Lee in view of Cederlund teaches the method of claim 8 as discussed above. The limitation of claim 10 is substantially similar to claim 3. Therefore, claim 10 is rejected based on the same analysis as claim 3.
Regarding claim 11; Lee in view of Cederlund teaches the method of claim 10 as discussed above. The limitation of claim 11 is substantially similar to claim 4. Therefore, claim 11 is rejected based on the same analysis as claim 4.
Regarding claim 13; Lee in view of Cederlund teaches the method of claim 8 as discussed above. The limitation of claim 13 is substantially similar to claim 6. Therefore, claim 13 is rejected based on the same analysis as claim 6.
Regarding claim 14; Lee in view of Cederlund teaches the method of claim 8 as discussed above. The limitation of claim 14 is substantially similar to claim 7. Therefore, claim 14 is rejected based on the same analysis as claim 7.
Regarding claim 15; Lee in view of Cederlund teaches a non-transitory computer-readable storage medium (Lee, a storage unit 570, Fig.5) storing one or more programs configured to be executed by one or more processors (Lee, a controller 510, Fig.5) of an electronic device (Lee, para. [0062]) that comprises one or more sensors and communication circuitry, the one or more programs including instructions for: obtaining, via a first subset of the one or more sensors, an image, wherein the image includes a display; obtaining, via a second subset of the one or more sensors, a gaze input; determining, using at least the gaze input and the image, a location on the display corresponding to the gaze input; obtaining, via the second subset of the one or more sensors, a gesture input; and transmitting information associated with the location on the display and the gesture input to an external electronic device using the communication circuitry (similar to the analysis of claim 1 above).
Regarding claim 17; Lee in view of Cederlund teaches the non-transitory computer-readable storage medium of claim 15 as discussed above. The limitation of claim 17 is substantially similar to claim 3. Therefore, claim 17 is rejected based on the same analysis as claim 3.
Regarding claim 18; Lee in view of Cederlund teaches the non-transitory computer-readable storage medium of claim 17 as discussed above. The limitation of claim 18 is substantially similar to claim 4. Therefore, claim 18 is rejected based on the same analysis as claim 4.
Regarding claim 20; Lee in view of Cederlund teaches the non-transitory computer-readable storage medium of claim 15 as discussed above. The limitation of claim 20 is substantially similar to claim 6. Therefore, claim 20 is rejected based on the same analysis as claim 6.
Regarding claim 21; Lee in view of Cederlund teaches the non-transitory computer-readable storage medium of claim 15 as discussed above. The limitation of claim 21 is substantially similar to claim 7. Therefore, claim 21 is rejected based on the same analysis as claim 7.
Claims 2, 9, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US Pub. 2013/0113738 A1) in view of Cederlund et al. (US Pub. 2015/0130740 A1) as applied to claims 1, 8, and 15 above, and further in view of Sydorenko et al. (US Pub. 2020/0394408 A1).
Regarding claim 2; Lee in view of Cederlund teaches the electronic device of claim 1 as discussed above.
Lee further teaches the electronic device has first and second opposing sides (Fig.1, the mobile terminal 120 has a front side and a rear side), wherein the first subset of the one or more sensors comprises a first camera on the first side (Fig.1, para. [0029], the mobile terminal 120 comprises a camera to capture the remote screen 111. It is understood that the camera would be disposed on the rear side of the mobile terminal 120).
Lee in view of Cederlund does not teach the second subset of the one or more sensors comprises a second camera on the second side.
Sydorenko teaches the second subset of the one or more sensors comprises a second camera on the second side (Figs. 2 and 3, para. [0102 and 0171], an electronic device 200/300 comprises a front camera 320 configured to determine eye-gaze position).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the system of Lee in view of Cederlund to include the teaching of Sydorenko of providing a front camera on an electronic device to detect an eye’s gaze. The motivation would have been in order to simplify the system.
Regarding claim 9; Lee in view of Cederlund teaches the method of claim 8 as discussed above. The limitation of claim 9 is substantially similar to claim 2. Therefore, claim 9 is rejected based on the same analysis as claim 2.
Regarding claim 16; Lee in view of Cederlund teaches the non-transitory computer-readable storage medium of claim 15 as discussed above. The limitation of claim 16 is substantially similar to claim 2. Therefore, claim 16 is rejected based on the same analysis as claim 2.
Claims 5, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US Pub. 2013/0113738 A1) in view of Cederlund et al. (US Pub. 2015/0130740 A1) as applied to claims 1, 8, and 15 above, and further in view of Ko et al. (US Pub. 2019/0075610 A1) and Greene (US Pub. 2017/0243474 A1).
Regarding claim 5; Lee in view of Cederlund teaches the electronic device of claim 1 as discussed above.
Lee in view of Cederlund does not teach obtaining a user input associated with an intent for interaction with the external electronic device, wherein obtaining the gaze input comprises obtaining the gaze input in response to obtaining the user input associated with the intent for interaction with the external electronic device and wherein obtaining the image comprises obtaining the image in response to obtaining the user input associated with the intent for interaction with the external electronic device; and in accordance with obtaining the user input associated with the intent for interaction with the external electronic device, transmitting, using the communication circuitry, an instruction to the external electronic device to suppress user input from an accessory electronic device.
Cederlund teaches obtaining the gaze input (see the analysis of claim 1 above). The motivation is the same as the rejection of claim 1.
Ko teaches obtaining a user input associated with an intent for interaction with the external electronic device (Fig.1, para. [0074], Ko discloses a method of pairing a mobile device 1000 with nearby electronic devices 210 through 230. A user may select an external electronic device to be controlled. For example, in Fig.11, para. [0145-0147], the mobile device 1000 may select an electronic device to pair with from among multiple electronic devices included in an obtained image. The mobile device 1000 may receive an input from a user to select an electronic device for pairing from between the remote controller 210 and the TV 240. For example, if an input to pair with the TV 240 is received from a user via the display unit 1400, the mobile device 1000 pairs with the TV 240).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the method of Lee in view of Cederlund for controlling content on remote screen to include the method of Ko of enabling a user to select an electronic device for pairing with a mobile device. Accordingly, in the system of Lee in view of Cederlund and Ko, the mobile terminal would receive an input of the user to select a remote apparatus among multiple remote apparatuses for pairing. Then, after establishing a connection between the mobile terminal and a remote apparatus, the user may perform a gaze input and a gesture input on the mobile terminal to control the user interface on the remote screen. Therefore, a combination of Lee, Cederlund, and Ko teaches “obtaining a user input associated with an intent for interaction with the external electronic device, wherein obtaining the gaze input comprises obtaining the gaze input in response to obtaining the user input associated with the intent for interaction with the external electronic device and wherein obtaining the image comprises obtaining the image in response to obtaining the user input associated with the intent for interaction with the external electronic device; and in accordance with obtaining the user input associated with the intent for interaction with the external electronic device”. The motivation would have been in order to enable the user to easily select a desired controllable device.
Greene teaches in accordance with obtaining the user input associated with the intent for interaction with the external electronic device, transmitting, using the communication circuitry, an instruction to the external electronic device to suppress user input from an accessory electronic device (Fig.2, para. [0030], Greene discloses a method of pairing a remote control with an A/V system 12. Specifically, the remote 30a is paired to the A/V system 12a, the remote 30b is paired to the A/V system 12b and the remote 30c is paired to the A/V system 12c. When a remote control 30 and an A/V system are paired, the A/V system 12 will only respond to commands from the particular remote control 30 with which it is paired. Therefore, when a user 28a presses a button on the remote control 30a to control the A/V system 12a, the radio signal which is transmitted carries through the entire network 10 and is received by the A/V systems 12b and 12c. The codes in the other A/V systems of the network, other than the one with which the remote control 30a is paired, are set to ignore all signals except from their own specific remote control they are paired with. Therefore, when a signal is output by a remote control 30a, it is received by the A/V systems 12b and 12c but they ignore these systems because they are not paired with that particular remote control. The A/V system 12a is paired to remote control 30a and therefore it responds to signals output by the remote control 30a).
At the time of invention was effectively filed, it would have been obvious to one of ordinary skill in the art to modify the system of Lee in view of Cederlund and Ko to include the teaching of Greene of enabling an A/V system to ignore all signals from a remote control that they are not paired with; and only responding to signals output from a paired remote control. The motivation would have been in order to reduce erroneous inputs.
Regarding claim 12; Lee in view of Cederlund teaches the method of claim 8 as discussed above. The limitation of claim 12 is substantially similar to claim 5. Therefore, claim 12 is rejected based on the same analysis as claim 5.
Regarding claim 19; Lee in view of Cederlund teaches the non-transitory computer-readable storage medium of claim 15 as discussed above. The limitation of claim 19 is substantially similar to claim 5. Therefore, claim 19 is rejected based on the same analysis as claim 5.
Inquiries
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NGUYEN H TRUONG whose telephone number is (571)270-1630. The examiner can normally be reached M-F: 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chanh Nguyen can be reached at 571-272-7772. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NGUYEN H TRUONG/Examiner, Art Unit 2623
/CHANH D NGUYEN/Supervisory Patent Examiner, Art Unit 2623
Prosecution Insights