CTNF 18/962,983 CTNF 100862 ETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Information Disclosure Statement 1. The information disclosure statements (IDS) submitted on 11/27/ are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements have been considered by the examiner. Drawings 06-22-06 AIA 2. The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: 1140 of ¶ [0191], line 9 . Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance. 06-22-07 AIA 3. The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: 340, 345, and 364 in Fig. 3B, t6 in Fig. 7B, and 1001 and 1002 in Fig. 10A . Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance. Claim Rejections - 35 USC § 102 07-07-aia AIA 07-07 4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. 07-15 AIA 5. Claim s 1, 7-8, 11, 16, an 19-20 are rejected under 35 U.S.C. 102( a)(1 ) as being anticipated by Lee et al. (US-2017/0185276-A1, hereinafter "Lee") . 6. As per claim 1, Lee discloses: A wearable device comprising: (Lee, [0065], “ The electronic device 100 described herein may a device wearable by a user. ”) a display; (Lee, [0065], “ The wearable device may include … a head-mounted device (HMD) … ” and [0088], “ The electronic device 100 may display the objects in various forms. For example, the electronic device 100 may display actual images of the objects using a live view … ”) memory comprising one or more storage media storing instructions; and (Lee, [0560], “ A method according to an exemplary embodiment may be embodied in the form of program instructions executable by various computing tools and recorded in a computer-readable recording medium. ... Examples of the computer-readable recording medium include magnetic media, such as a hard disk, a floppy disk, and a magnetic tape ... specially configured to store and execute the program instructions. ” and [0531], “ The storage 150 may store programs for processing and control of the processor 170, and may perform input and/or output of data (e.g., attribute information of an object, communication connection information, etc.). ”) at least one processor comprising processing circuitry, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (Lee, [0545], “ The processor 170 usually controls overall operation of the electronic device 100. For example, by controlling the programs stored in the storage 150, the processor 170 may control all of the output transmitter 110, the communication interface 120, the user interface 130, the A/V input receiver 140, the storage 150, the sensors 160, and so on. ”) receive an input indicating selection of a first external electronic device and a second external electronic device among a plurality of external electronic devices viewable through the display; (Lee, [0087], “ According to an exemplary embodiment, when three or more objects are recognized through the camera or a communication interface, the electronic device 100 may display the recognized objects on the screen and receive an input for selecting a first object and a second object among the recognized objects from the user. ” and [0143], “ According to an exemplary embodiment, the electronic device 100 may receive an input for selecting the TV and the light fixture in the live view. ” and [0067], “ When the user selects identification information of the light fixture 300 to control among the detected devices, the electronic device 100 may perform a pairing procedure with the light fixture 300, and then display a graphical user interface (GUI) for adjusting the illuminance of the light fixture 300. ” and [0142], “ The electronic device 100 may mark objects selectable by the user in the live view. For example, when objects selectable by the user are the TV, the light fixture, and the audio device, the electronic device 100 may display a first icon close to the TV, a second icon close to the light fixture, and a third icon close to the audio device. ”) based on the input, identify third external electronic devices in a category including both the first external electronic device and the second external electronic device, and identify the first external electronic device and the second external electronic device along with the third external electronic devices; (Lee, [0087], “ According to an exemplary embodiment, when three or more objects are recognized through the camera or a communication interface, the electronic device 100 may display the recognized objects on the screen and receive an input for selecting a first object and a second object among the recognized objects from the user. ” and [0361], “ For example, referring to FIG. 27, the server 2600 may determine product groups (e.g., a TV, an air conditioner, a vacuum cleaner, etc.) as categories 2710. ” and [0142], “ The electronic device 100 may mark objects selectable by the user in the live view. For example, when objects selectable by the user are the TV, the light fixture, and the audio device, the electronic device 100 may display a first icon close to the TV, a second icon close to the light fixture, and a third icon close to the audio device. ” and [0331], “ Referring to FIG. 22, the electronic device 100 may display a live view including a TV, a light fixture, an audio device 2224, a washing machine 2226, and a robot vacuum cleaner 2228. Through image processing of an image corresponding to the live view, the electronic device 100 may identify positions of the TV, the light fixture, the audio device, the robot vacuum cleaner, and the washing machine in the live view. Also, through short-range communication, the electronic device 100 may identify the positions of the TV, the light fixture, the audio device, the robot vacuum cleaner, and the washing machine in the live view. ”) display a first visual object and a second visual object respectively indicating that the first external electronic device and the second external electronic device are selected; and (Lee, [0087], “ According to an exemplary embodiment, when three or more objects are recognized through the camera or a communication interface, the electronic device 100 may display the recognized objects on the screen and receive an input for selecting a first object and a second object among the recognized objects from the user. ” and [0088], “ The electronic device 100 may display the objects in various forms. For example, the electronic device 100 may display actual images of the objects using a live view which shows a subject recognized through the camera. ” and [0142], “ The electronic device 100 may mark objects selectable by the user in the live view. For example, when objects selectable by the user are the TV, the light fixture, and the audio device, the electronic device 100 may display a first icon close to the TV, a second icon close to the light fixture, and a third icon close to the audio device. ” and [0143], “ According to an exemplary embodiment, the electronic device 100 may receive an input for selecting the TV and the light fixture in the live view. ” and [0158], “ For example, the images corresponding to the plurality of objects on a one-to-one basis may be icon images, thumbnail images, signs, figures, text images, etc., but are not limited thereto. ”; Examiner’s note: As disclosed in Lee [0158], the visual object could be represented by “icon images, thumbnail images, signs, figures, text images, etc.”) display at least one third visual object guiding selection of at least one fourth external electronic device among the third external electronic devices. (Lee, [0087], “ According to an exemplary embodiment, when three or more objects are recognized through the camera or a communication interface, the electronic device 100 may display the recognized objects on the screen and receive an input for selecting a first object and a second object among the recognized objects from the user. ” and [0089], “ According to an exemplary embodiment, the electronic device 100 may generate an object map including virtual images (e.g., icon images, text images, etc.) corresponding to objects and display the object map on the screen. ” and [0166], “ According to an exemplary embodiment, the electronic device 100 may display an object map 700 including virtual images of surrounding objects. For example, the object map 700 may include images (e.g., a TV icon 701, a light fixture icon 702, a dog 704, an audio device icon 706, a window 708, a vacuum cleaner icon 710, a computer icon 712, an image 714 of the electronic device 100, and a microwave icon 716) of objects recognized, e.g., sensed, through short-range communication. ” and [0331], “ Referring to FIG. 22, the electronic device 100 may display a live view including a TV, a light fixture, an audio device 2224, a washing machine 2226, and a robot vacuum cleaner 2228. Through image processing of an image corresponding to the live view, the electronic device 100 may identify positions of the TV, the light fixture, the audio device, the robot vacuum cleaner, and the washing machine in the live view. Also, through short-range communication, the electronic device 100 may identify the positions of the TV, the light fixture, the audio device, the robot vacuum cleaner, and the washing machine in the live view. ” and [0142], “ The electronic device 100 may mark objects selectable by the user in the live view. For example, when objects selectable by the user are the TV, the light fixture, and the audio device, the electronic device 100 may display a first icon close to the TV, a second icon close to the light fixture, and a third icon close to the audio device. ” and [0316], “ The electronic device 100 may transmit a concert video file of the first user (e.g., Jane) or a screen showing the concert video of the first user (e.g., Jane) to the third TV 2030 of the third user. ” and [0145], “ According to an exemplary embodiment, the electronic device 100 may receive an input for selecting three or more objects and control two or more objects among the three or more objects. ” and [0158], “ For example, the images corresponding to the plurality of objects on a one-to-one basis may be icon images, thumbnail images, signs, figures, text images, etc., but are not limited thereto. ”; Examiner’s note: As disclosed in Lee [0158], the visual object could be represented by “icon images, thumbnail images, signs, figures, text images, etc.”) 7. As per claim 7, Lee discloses: The wearable device of claim 1, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (See rejection for claim 1.) based on the input, receive first data obtained by the first external electronic device and second data obtained by the second external electronic device; (Lee, [0087], “ According to an exemplary embodiment, when three or more objects are recognized through the camera or a communication interface, the electronic device 100 may display the recognized objects on the screen and receive an input for selecting a first object and a second object among the recognized objects from the user. ” and [0177], “ The electronic device 100 may establish the first communication link and the second communication link by exchanging identification information (e.g., device IDs, media access control (MAC) addresses, device names, etc.), function information (e.g., support for BLE, Bluetooth, Ant+, Wi-Fi, and NFC), information on a preferred communication method (e.g., Bluetooth), etc. with the first object 801 and the second object 802. ”) display, on the display, a fourth visual object based on a combination of the first data and the second data. (Lee, [0337], “ For example, when two attributes correspond to the first object and two attributes correspond to the second object, there may be four operations corresponding to a combination of the attributes of the first object and the attributes of the second object. Therefore, the electronic device 100 may provide a list of a plurality of operations corresponding to the first object and the second object which are selected by the user. For example, the electronic device 100 may display all of the four operations or recommended two of the four operations. ” and [0158], “ For example, the images corresponding to the plurality of objects on a one-to-one basis may be icon images, thumbnail images, signs, figures, text images, etc., but are not limited thereto. ”; Examiner’s note: As disclosed in Lee [0158], the visual object could be represented by “icon images, thumbnail images, signs, figures, text images, etc.” The four operations displayed in [0337] represents a combination of all functional attributes between the first object and second object (electronic devices).) 8. As per claim 8, Lee discloses: The wearable device of claim 1, wherein the input is a first input, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (See rejection for claim 1.) based on a second input indicating selection of the at least one third visual object, identify one or more functions applicable to the third external electronic devices; and (Lee, [0087], “ According to an exemplary embodiment, when three or more objects are recognized through the camera or a communication interface, the electronic device 100 may display the recognized objects on the screen and receive an input for selecting a first object and a second object among the recognized objects from the user. ” and [0144], “ Referring to a second screen 520, the electronic device 100 may receive an input of drawing a closed figure (e.g., a circle, a rectangle, etc.). The electronic device 100 may select an object through which the closed figure passes and an object in the closed figure. For example, when the user draws a circle passing through the TV, the light fixture, and the audio device on the live view, the electronic device 100 may recognize that the user has selected the TV, the light fixture, and the audio device. ” and [0357], “ Meanwhile, according to an exemplary embodiment, object recognition model information may include information on applications (or controllers) linked to each of a plurality of objects. For example, object recognition model information may include information on a third application (e.g., a music player application) linked to a third object (e.g., a speaker). ” and [0213], “ For example, the electronic device 100 may receive a drag input of touching and dragging the TV 1001 to the light fixture 1002. ” and [0122]-[0123], “ For example, the user input for selecting the first object and the second object may be varied. For example, the user input for selecting the first object and the second object may be at least one of a touch input, a voice input, an ocular input, and a bending input, but is not limited thereto. ... For example, a touch input stated in an exemplary embodiment may be a tap, a touch and hold, a double tap, a drag, panning, a flick, a drag and drop, or so on. ” and [0133], “ Also, the electronic device 100 may receive an input of tapping, double tapping, or touching and holding each of the first object and the second object, but the touch input is not limited thereto. ”; Examiner’s note: There are many different input methods disclosed by Lee in [0122]-[0123]. For instance, a user could drag the TV to the light fixture or draw a closed figure, such as a circle, around the TV and the light fixture, selecting those two devices with a single input. Then the user could utilize a “touch and hold” as a second input to select a third device, such as the audio device.) display, within the display, one or more fourth visual objects to execute the one or more functions. (Lee, [0337], “ For example, when two attributes correspond to the first object and two attributes correspond to the second object, there may be four operations corresponding to a combination of the attributes of the first object and the attributes of the second object. Therefore, the electronic device 100 may provide a list of a plurality of operations corresponding to the first object and the second object which are selected by the user. For example, the electronic device 100 may display all of the four operations or recommended two of the four operations. ” and [0357], “ Meanwhile, according to an exemplary embodiment, object recognition model information may include information on applications (or controllers) linked to each of a plurality of objects. For example, object recognition model information may include information on a third application (e.g., a music player application) linked to a third object (e.g., a speaker). ” and [0158], “ For example, the images corresponding to the plurality of objects on a one-to-one basis may be icon images, thumbnail images, signs, figures, text images, etc., but are not limited thereto. ”; Examiner’s note: As disclosed in Lee [0158], the visual object could be represented by “icon images, thumbnail images, signs, figures, text images, etc.” The four operations displayed in [0337] represents a combination of all functional attributes between the first object and second object (electronic devices). Because of the inclusion of a third device, the displayed operation list items could display even more than four operations depending on the functional attributes of the third device.) 9. Claim 11 is similar in scope to claim 1 except for a different limitation that Lee discloses: A method of a wearable device, the method comprising: receiving a first input indicating selection of a first external electronic device and a second external electronic device among a plurality of external electronic devices viewable through a display of the wearable device; (Lee, [0087], “ According to an exemplary embodiment, when three or more objects are recognized through the camera or a communication interface, the electronic device 100 may display the recognized objects on the screen and receive an input for selecting a first object and a second object among the recognized objects from the user. ” and [0143], “ According to an exemplary embodiment, the electronic device 100 may receive an input for selecting the TV and the light fixture in the live view. ” And [0065], “ The electronic device 100 described herein may a device wearable by a user. ” and “ The wearable device may include … a head-mounted device (HMD) … ” and [0088], “ The electronic device 100 may display the objects in various forms. For example, the electronic device 100 may display actual images of the objects using a live view … ”) 10. As per claim 16, Lee discloses: A wearable device comprising: (Lee, [0065], “ The electronic device 100 described herein may a device wearable by a user. ”) a display; (Lee, [0065], “ The wearable device may include … a head-mounted device (HMD) … ” and [0088], “ The electronic device 100 may display the objects in various forms. For example, the electronic device 100 may display actual images of the objects using a live view … ”) memory comprising one or more storage media storing instructions; and (Lee, [0560], “ A method according to an exemplary embodiment may be embodied in the form of program instructions executable by various computing tools and recorded in a computer-readable recording medium. ... Examples of the computer-readable recording medium include magnetic media, such as a hard disk, a floppy disk, and a magnetic tape ... specially configured to store and execute the program instructions. ” and [0531], “ The storage 150 may store programs for processing and control of the processor 170, and may perform input and/or output of data (e.g., attribute information of an object, communication connection information, etc.). ”) at least one processor comprising processing circuitry, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (Lee, [0545], “ The processor 170 usually controls overall operation of the electronic device 100. For example, by controlling the programs stored in the storage 150, the processor 170 may control all of the output transmitter 110, the communication interface 120, the user interface 130, the A/V input receiver 140, the storage 150, the sensors 160, and so on. ”) receive a first input indicating selection of one or more first external electronic devices among a plurality of external electronic devices viewable through the display; (Lee, Fig. 10A-C (see figures below); [0087], “ According to an exemplary embodiment, when three or more objects are recognized through the camera or a communication interface, the electronic device 100 may display the recognized objects on the screen and receive an input for selecting a first object and a second object among the recognized objects from the user. ” and [0143], “ According to an exemplary embodiment, the electronic device 100 may receive an input for selecting the TV and the light fixture in the live view. ”) based on the first input, identify one or more first functions applicable to the one or more first external electronic devices; (Lee, Fig. 10A-C (see figures below); [0091]-[0092], “ In operation S220, the electronic device 100 may identify attributes of the first object and attributes of the second object. Here, attributes of an object may include an identifier (ID) of the object, a type of the object, a function provided by the object, a current state of the object, etc., but are not limited thereto. Attributes of an object may vary according to a type of the object. ” and [0106], “ According to an exemplary embodiment, the electronic device 100 may determine an operation (referred to as a control operation below) to be performed by the selected object based on attributes of the first object and attributes of the second object. For example, the control operation may be one of functions provided by the selected object. For example, when the target object is a TV, the control operation may be one of “channel change,” “video on demand (VOD) playing,” “brightness adjustment,” “volume adjustment,” and “application execution,” but is not limited thereto. Also, when the target object is a light fixture, the control operation may be one of “brightness adjustment,” “color adjustment,” and “adjustment of flickering periods,” but is not limited thereto. ” and [0210], “ FIGS. 10A, 10B and 10C are diagrams illustrating an operation of an electronic device controlling a light fixture based on an attribute of a TV and an attribute of the light fixture according to an exemplary embodiment. ”) display, within the display, one or more visual objects for executing the one or more first functions; (Lee, Fig. 10A-C (see figures below); [0101]-[0102], “ According to an exemplary embodiment, the electronic device 100 may select one of a plurality of attributes corresponding to an object based on a user input. For example, the electronic device 100 may display attributes corresponding to each of a plurality of objects together with the plurality of objects on the screen, and receive a user input for selecting one of the displayed attributes. ... In operation 5230, the electronic device 100 may select an object to control between the first object and the second object based on the attributes of the first object and the attributes of the second object. ” and [0158], “ For example, the images corresponding to the plurality of objects on a one-to-one basis may be icon images, thumbnail images, signs, figures, text images, etc., but are not limited thereto. ”; Examiner’s note: As disclosed in Lee [0158], the visual object could be represented by “icon images, thumbnail images, signs, figures, text images, etc.”) in a state that the one or more visual objects are displayed, identify, based on a second input indicating selection of at least one second external electronic device, at least one second function applicable to the at least one second external electronic device; (Lee, Fig. 10A-C (see figures below); [0106], “ According to an exemplary embodiment, the electronic device 100 may determine an operation (referred to as a control operation below) to be performed by the selected object based on attributes of the first object and attributes of the second object. For example, the control operation may be one of functions provided by the selected object. For example, when the target object is a TV, the control operation may be one of “channel change,” “video on demand (VOD) playing,” “brightness adjustment,” “volume adjustment,” and “application execution,” but is not limited thereto. Also, when the target object is a light fixture, the control operation may be one of “brightness adjustment,” “color adjustment,” and “adjustment of flickering periods,” but is not limited thereto. ” and [0210], “ FIGS. 10A, 10B and 10C are diagrams illustrating an operation of an electronic device controlling a light fixture based on an attribute of a TV and an attribute of the light fixture according to an exemplary embodiment. ” and [0212]-[0213], “ For example, the electronic device 100 may identify a TV 1001, a light fixture 1002, an audio device 2224, a washing machine 2226, and a robot vacuum cleaner 2228. Also, the electronic device 100 may display state information (e.g., Watching, Volume: 60) of the TV 1001, state information (e.g., Brightness: 70%) of the light fixture 1002, state information (e.g., Waiting, Volume: 60) of the audio device 2224, state information (e.g., Washing, Remaining time: 30 minutes) of the washing machine 2226, state information (e.g., Cleaning, Battery: 60% left) of the robot vacuum cleaner 2228, and so on. When the user wants to view a movie played on the TV 1001 in a dark environment, the electronic device 100 may receive a user input for selecting the TV 1001 and the light fixture 1002 through the live view. For example, the electronic device 100 may receive a drag input of touching and dragging the TV 1001 to the light fixture 1002. ” and [0158], “ For example, the images corresponding to the plurality of objects on a one-to-one basis may be icon images, thumbnail images, signs, figures, text images, etc., but are not limited thereto. ”; Examiner’s note: As disclosed in Lee [0158], the visual object could be represented by “icon images, thumbnail images, signs, figures, text images, etc.” In [0212] and Fig. 10A, Lee discloses multiple electronic devices that are identified and displays a status of each based on their identified attributes. In [0213], a user selects both the TV and light fixture. The TV has its own functional attributes (“first functions”) and the light fixture has its own functional attributes (“second functions”).) cease to, among the one or more visual objects, displaying at least one visual object with respect to a function different from the at least one second function. (Lee, Fig. 10A-C (see figures below); [0213]-[0215], “ When the user wants to view a movie played on the TV 1001 in a dark environment, the electronic device 100 may receive a user input for selecting the TV 1001 and the light fixture 1002 through the live view. ... For example, the electronic device 100 may determine the light fixture 1002 as a target object, and determine “brightness: set to 10%” as a target operation. Referring to FIG. 10B, the electronic device 100 may display a pop-up window 1003 which requests confirmation of the target object (light fixture 1002) and the target operation (“brightness: set to 10%”) determined based on the attribute of the TV 1001 and the attribute of the light fixture 1002. For example, the electronic device 100 may display the pop-up window including the message “Do you want to adjust illumination for the movie mode?”. ” and [0101], “ According to an exemplary embodiment, the electronic device 100 may select one of a plurality of attributes corresponding to an object based on a user input. For example, the electronic device 100 may display attributes corresponding to each of a plurality of objects together with the plurality of objects on the screen, and receive a user input for selecting one of the displayed attributes. ” and [0337], “ For example, when two attributes correspond to the first object and two attributes correspond to the second object, there may be four operations corresponding to a combination of the attributes of the first object and the attributes of the second object. Therefore, the electronic device 100 may provide a list of a plurality of operations corresponding to the first object and the second object which are selected by the user. For example, the electronic device 100 may display all of the four operations or recommended two of the four operations. ” and [0158], “ For example, the images corresponding to the plurality of objects on a one-to-one basis may be icon images, thumbnail images, signs, figures, text images, etc., but are not limited thereto. ”; Examiner’s note: As disclosed in Lee [0158], the visual object could be represented by “icon images, thumbnail images, signs, figures, text images, etc.” In [0212] and Fig. 10A, Lee discloses multiple electronic devices that are identified and displays a status of each based on their identified attributes. After the user has selected both the TV and light fixture, a popup window is displayed shown in Fig. 10B. Fig. 10B shows a prompt related to adjusting the light fixture and ceases to display other attributes/functions of other electronic devices unrelated to the TV and light fixture. Fig. 10C shows the result after performing the function related to the TV and light fixture (and their attribute status) and ceases to display other attributes/functions of other electronic devices unrelated to the TV and light fixture. In addition [0101] discloses the ability for a user to select a single attribute for display of an object (while ceasing display of other attributes). Lastly, [0337] discloses that the device may not display a list of all possible operations but rather a subset of recommended operations (ceasing display of all possible operations).) PNG media_image1.png 769 1099 media_image1.png Greyscale Lee et al. (US-2017/0185276-A1), Fig. 10A: Lee discloses multiple electronic devices that are identified and displays a status of each based on their identified attributes. PNG media_image2.png 769 1099 media_image2.png Greyscale Lee et al. (US-2017/0185276-A1), Fig. 10B: A popup window is shown after a user has selected both the TV and light fixture. A prompt is shown related to adjusting the light fixture (and ceases display of other attributes/functions). PNG media_image3.png 769 1099 media_image3.png Greyscale Lee et al. (US-2017/0185276-A1), Fig. 10C: After performing the function related to the TV and light fixture, their status continues to be shown and is updated (while removing attributes/functions of other electronic devices). 11. As per claim 19, Lee discloses: The wearable device of claim 16, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (See rejection for claim 16.) receive a third input indicating selection of one of the one or more visual objects; and (Lee, [0142], “ The electronic device 100 may mark objects selectable by the user in the live view. For example, when objects selectable by the user are the TV, the light fixture, and the audio device, the electronic device 100 may display a first icon close to the TV, a second icon close to the light fixture, and a third icon close to the audio device. ” and [0122]-[0123], “ For example, the user input for selecting the first object and the second object may be varied. For example, the user input for selecting the first object and the second object may be at least one of a touch input, a voice input, an ocular input, and a bending input, but is not limited thereto. ... For example, a touch input stated in an exemplary embodiment may be a tap, a touch and hold, a double tap, a drag, panning, a flick, a drag and drop, or so on. ” and [0133], “ Also, the electronic device 100 may receive an input of tapping, double tapping, or touching and holding each of the first object and the second object, but the touch input is not limited thereto. ”; Examiner’s note: There are many different input methods disclosed by Lee in [0122]-[0123]. For instance, a user could utilize “touch and hold” as three individual inputs as a means of selecting “three visual objects” representing electronic devices.) based on the third input, request to the one or more first external electronic devices to execute a function corresponding to a visual object selected by the third input from among the one or more first functions. (Lee, [0145], “ According to an exemplary embodiment, the electronic device 100 may receive an input for selecting three or more objects and control two or more objects among the three or more objects. For example, the electronic device 100 may determine the light fixture and the audio device as target objects based on attributes of the TV, attributes of the light fixture, and attributes of the audio device, transmit a first operation signal which instructs to change illuminance according to content played on the TV to the light fixture, and transmit a second operation signal which instructs to adjust the volume according to the played content to the audio device. ” and [0158], “ For example, the images corresponding to the plurality of objects on a one-to-one basis may be icon images, thumbnail images, signs, figures, text images, etc., but are not limited thereto. ”; Examiner’s note: As disclosed in Lee [0158], the visual object could be represented by “icon images, thumbnail images, signs, figures, text images, etc.”) 12. As per claim 20, Lee discloses: The wearable device of claim 16, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (See rejection for claim 16.) based on the at least one second function identified based on the second input, maintain displaying of at least one visual object corresponding to the at least one second function. (Lee, Fig. 10A-C; [0213]-[0215], “ When the user wants to view a movie played on the TV 1001 in a dark environment, the electronic device 100 may receive a user input for selecting the TV 1001 and the light fixture 1002 through the live view. ... For example, the electronic device 100 may determine the light fixture 1002 as a target object, and determine “brightness: set to 10%” as a target operation. Referring to FIG. 10B, the electronic device 100 may display a pop-up window 1003 which requests confirmation of the target object (light fixture 1002) and the target operation (“brightness: set to 10%”) determined based on the attribute of the TV 1001 and the attribute of the light fixture 1002. For example, the electronic device 100 may display the pop-up window including the message “Do you want to adjust illumination for the movie mode?”. ” and [0087], “ According to an exemplary embodiment, when three or more objects are recognized through the camera or a communication interface, the electronic device 100 may display the recognized objects on the screen and receive an input for selecting a first object and a second object among the recognized objects from the user. ” and [0101], “ According to an exemplary embodiment, the electronic device 100 may select one of a plurality of attributes corresponding to an object based on a user input. For example, the electronic device 100 may display attributes corresponding to each of a plurality of objects together with the plurality of objects on the screen, and receive a user input for selecting one of the displayed attributes. ”; Examiner’s note: In [0212] and Fig. 10A, Lee discloses multiple electronic devices that are identified and displays a status of each based on their identified attributes. After the user has selected both the TV and light fixture, a popup window is displayed shown in Fig. 10B. Fig. 10B shows a prompt related to adjusting the light fixture. Fig. 10C shows the result after performing the function related to the TV and light fixture and maintains display of their attribute status. In addition [0101] discloses the ability for a user to select a single attribute for display of an object (while ceasing display of other attributes).) Claim Rejections - 35 USC § 103 07-20-aia AIA 13. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA 14. Claim s 2-4, 6, 12-15, and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US-2017/0185276-A1, hereinafter "Lee") in view of Kim et al. (US-2020/0380771-A1, hereinafter "Kim") . 15. As per claim 2, Lee discloses: The wearable device of claim 1, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (See rejection for claim 1.) based on the input, identify a first category of the first external electronic device in accordance with first [[metadata]] with respect to the first external electronic device, and identify a second category of the second external electronic device in accordance with second [[metadata]] with respect to the second external electronic device; and (Lee, [0356], “ A model for recognizing an object (which will be referred to as an “object recognition model” below) may include images for identifying the object. For example, a first object recognition model for recognizing a first object (e.g., a TV) may include first images (e.g., TV images) corresponding to the first object, and a second object recognition model for recognizing a second object (e.g., a refrigerator) may include second images (e.g., refrigerator images) corresponding to the second object. ” and [0358]-[0361], “ According to an exemplary embodiment, object recognition model information may be generated by the electronic device 100 or a server. ... FIG. 26 is a sequence diagram illustrating a method of a server generating object recognition model information according to an exemplary embodiment. In operation S2610, a server 2600 may determine a plurality of categories and a plurality of keywords. For example, referring to FIG. 27, the server 2600 may determine product groups (e.g., a TV, an air conditioner, a vacuum cleaner, etc.) as categories 2710. ” and [0177], “ The electronic device 100 may establish the first communication link and the second communication link by exchanging identification information (e.g., device IDs, media access control (MAC) addresses, device names, etc.), function information (e.g., support for BLE, Bluetooth, Ant+, Wi-Fi, and NFC), information on a preferred communication method (e.g., Bluetooth), etc. with the first object 801 and the second object 802. ”; Examiner’s note: As disclosed by Lee in [0356], the images of objects serve as an input to the server for the “object recognition model.” Based on this input, categories can be determined for the objects. Also, as disclosed in [0177], the data transmitted provides information about the identified objects (devices) such as device name and function.) based on the first category and the second category, display executable objects to control the first external electronic device and the second external electronic device. (Lee, [0515]-[0516], “ The display 111 may display at least one attribute of a first object and a second object. For example, the display 111 may display an icon, a sign, or text corresponding to the at least one attribute. The display 111 may display a recommended operation list including a plurality of operations determined based on an attribute of the first object and an attribute of the second object. Also, the display 111 may display state information corresponding to the plurality of objects. ” and [0106], “ According to an exemplary embodiment, the electronic device 100 may determine an operation (referred to as a control operation below) to be performed by the selected object based on attributes of the first object and attributes of the second object. For example, the control operation may be one of functions provided by the selected object. For example, when the target object is a TV, the control operation may be one of “channel change,” “video on demand (VOD) playing,” “brightness adjustment,” “volume adjustment,” and “application execution,” but is not limited thereto. Also, when the target object is a light fixture, the control operation may be one of “brightness adjustment,” “color adjustment,” and “adjustment of flickering periods,” but is not limited thereto. ” and [0361], “ For example, referring to FIG. 27, the server 2600 may determine product groups (e.g., a TV, an air conditioner, a vacuum cleaner, etc.) as categories 2710. ” and [0337], “ For example, when two attributes correspond to the first object and two attributes correspond to the second object, there may be four operations corresponding to a combination of the attributes of the first object and the attributes of the second object. Therefore, the electronic device 100 may provide a list of a plurality of operations corresponding to the first object and the second object which are selected by the user. For example, the electronic device 100 may display all of the four operations or recommended two of the four operations. ”) 16. Lee doesn't explicitly disclose but Kim discloses: [[based on the input, identify a first category of the first external electronic device in accordance with first]] metadata [[with respect to the first external electronic device, and identify a second category of the second external electronic device in accordance with second]] metadata [[with respect to the second external electronic device; and]] (Kim, [0069], “ According to an embodiment, the electronic device 100 may identify a type of the target device in order to acquire the virtual object data from the three-dimensional image data of the target device. Here, the type of the target device may be information for identifying a category to which the target device belongs. The type of the device according to an embodiment of the disclosure may include a product group, a model name, etc. of the target device. For example, when the target device is a refrigerator, the type may include ‘electronic device’, ‘home appliance’, ‘refrigerator’, ‘2-door refrigerator’, ‘IRS82M6000’, etc. The electronic device 100 may use geometric data, attribute data, metadata, file name, etc. of the target device included in the three-dimensional image data, in order to identify the type of the target device. ”) 17. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the wearable device of claim 1 of Lee to include the disclosure of utilizing metadata to identify electronic devices, of Kim. The motivation for this modification could have been to utilize a common data format for distribution of electronic device information and functions. As metadata is frequently ubiquitous in various environments, it would simplify communication between devices and allow for easy addition of new devices to control. 18. As per claim 3, Lee in view of Kim discloses: The wearable device of claim 2, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (See rejection for claim 2.) identify one or more functions commonly included in a first set of functions applicable to an external electronic device in the first category and in a second set of functions applicable to an external object in the second category; and (Lee, [0358]-[0361], “ According to an exemplary embodiment, object recognition model information may be generated by the electronic device 100 or a server. ... FIG. 26 is a sequence diagram illustrating a method of a server generating object recognition model information according to an exemplary embodiment. In operation S2610, a server 2600 may determine a plurality of categories and a plurality of keywords. For example, referring to FIG. 27, the server 2600 may determine product groups (e.g., a TV, an air conditioner, a vacuum cleaner, etc.) as categories 2710. ” and [0357], “ Meanwhile, according to an exemplary embodiment, object recognition model information may include information on applications (or controllers) linked to each of a plurality of objects. For example, object recognition model information may include information on a third application (e.g., a music player application) linked to a third object (e.g., a speaker). ” and [0364], “ According to an exemplary embodiment, the server 2600 may search for an application or a controller corresponding to a category and a keyword. For example, when the category 2710 is “TV” and the keyword 2720 is “living room,” the server 2600 may search a website for a controller corresponding to a living room TV. Also, when the category 2710 is “water bottle” and the keyword 2720 is “smart,” the server 2600 may search a website for an application of a smart water bottle. ” and [0370], “ According to an exemplary embodiment, the server 2600 may generate object recognition model information by matching each of the plurality of object recognition models and application (or controller) information with each other. For example, the object recognition model information generated by the server 2600 may include first matching information obtained by matching an object recognition model of the living room TV with controller information for controlling the living room TV, second matching information obtained by matching an object recognition model of a refrigerator with refrigerator management application information, and so on. ”; Examiner’s note: As noted by Lee in [0358] a server generates a “object recognition model” and utilizes a plurality of categories as part of that process. The object recognition model can include information on applications (controllers) for a particular category of device. For instance [0357] discloses a music player application for a speaker and [0364] discloses a controller for a TV.) display, within the display, the executable objects to execute the one or more functions. (Lee, [0404]-[0405], “ According to an exemplary embodiment, the electronic device 100 may display a list of applications installed thereon, and receive an input of selecting at least one application in the application list. According to an exemplary embodiment, the electronic device 100 may identify the first object by analyzing the image of the first object and recommend at least one application corresponding to the first object. The electronic device 100 may receive an input of selecting the recommended application. ” and [0370], “ According to an exemplary embodiment, the server 2600 may generate object recognition model information by matching each of the plurality of object recognition models and application (or controller) information with each other. For example, the object recognition model information generated by the server 2600 may include first matching information obtained by matching an object recognition model of the living room TV with controller information for controlling the living room TV, second matching information obtained by matching an object recognition model of a refrigerator with refrigerator management application information, and so on. ”) 19. As per claim 4, Lee in view of Kim discloses: The wearable device of claim 2, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (See rejection for claim 2.) based on identifying the second category being different from the first category, identify a third category in which the third external electronic devices are included, based on an upper category of each of the first category of the first external electronic device and the second category of the second external electronic device. (Kim, Fig. 8 and Kim, [0069], “ According to an embodiment, the electronic device 100 may identify a type of the target device … Here, the type of the target device may be information for identifying a category to which the target device belongs. For example, when the target device is a refrigerator, the type may include ‘electronic device’, ‘home appliance’, ‘refrigerator’, ‘2-door refrigerator’, ‘IRS82M6000’, etc. ” and Kim, [0129], “ The database may include a plurality of orders based on the data field or the class. A type field of each order may include a plurality of elements. In FIG. 8, a database having a major classification, a middle classification, and a minor classification is shown as an example. In addition, FIG. 8 shows that ‘washing machine’, that is, an element of the major classification tier includes ‘Front-load’, ‘Top-load’, and ‘2-in-1’ as elements of the middle classification tier. Also, ‘Front-load’, that is, the element of the middle classification tier, includes model names ‘WF7500’, ‘WW6800’, and ‘WF5300’ as elements of the minor classification tier, as an example. However, the above example is only provided to describe the configuration method of the database, and one of ordinary skill in the art would appreciate that the hierarchical structure and elements in the database are not limited to the above example. ” and Lee, [0360]-[0361], “ In operation S2610, a server 2600 may determine a plurality of categories and a plurality of keywords. For example, referring to FIG. 27, the server 2600 may determine product groups (e.g., a TV, an air conditioner, a vacuum cleaner, etc.) as categories 2710. ”; Examiner’s note: As disclosed by Lee in Fig. 27 and [0361], the upper category for all of the devices is listed as a “product group.” Lee also discloses a plurality of categories in [0360]. As disclosed by Kim in [0129], the categories are divided into three subcategories labeled as “major classification, middle classification, and minor classification.” The major classification group matches the categories of Lee, such as “washing machine” and “refrigerator.” Kim then additionally has subcategories below the major classification group that defines the more specific nature of the device and its exact product model.) 20. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the wearable device of claim 2 of Lee to include the disclosure of further organizing categories with an upper category and subcategories, of Kim. The motivation for this modification could have been to further allow organization of different electronic devices based on their type. In a situation where there are many electronic devices, the upper category and subcategories would help to make it easier to filter out a device by its function or type. Particularly in a graphical user interface, a desired electronic device would be easier to find in organized categories rather than in a flat list of devices. 21. As per claim 6, Lee in view of Kim discloses: The wearable device of claim 2, wherein the input is a first input, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to, (See rejection for claim 2.) based on a second input indicating selection of one object among the executable objects, request, to the first external electronic device and the second external electronic device, execution of a function mapped to the object selected by the second input. (Lee, Fig. 10A-C; [0213]-[0215], “ When the user wants to view a movie played on the TV 1001 in a dark environment, the electronic device 100 may receive a user input for selecting the TV 1001 and the light fixture 1002 through the live view. ... For example, the electronic device 100 may determine the light fixture 1002 as a target object, and determine “brightness: set to 10%” as a target operation. Referring to FIG. 10B, the electronic device 100 may display a pop-up window 1003 which requests confirmation of the target object (light fixture 1002) and the target operation (“brightness: set to 10%”) determined based on the attribute of the TV 1001 and the attribute of the light fixture 1002. For example, the electronic device 100 may display the pop-up window including the message “Do you want to adjust illumination for the movie mode?”. ” and [0133], “ Also, the electronic device 100 may receive an input of tapping, double tapping, or touching and holding each of the first object and the second object, but the touch input is not limited thereto. ”; Examiner’s note: There are many different input methods disclosed by Lee in [0122]-[0123]. For instance, a user could utilize “touch and hold” to select the TV and light fixture as two inputs. Then, Fig. 10A-C and [0213]-[0215] discloses executing of a function related to both the TV and light fixture. The brightness of the light fixture is set so that the TV can play a movie. Fig. 10C shows the result after performing the function related to the TV and light fixture and displays their attribute status.) 22. Claim 12, which is similar in scope to dependent claim 2 and independent claim 11, is thus rejected under the same rationale as described above. The motivation for this modification is the same as claim 2. 23. Claim 13, which is similar in scope to dependent claims 3 and 12 and independent claim 11, is thus rejected under the same rationale as described above. 24. Claim 14 is similar in scope to claims 4, 11, and 12 except for an additional limitation that Lee in view of Kim discloses: based on identifying the third category that the third external electronic devices are included, displaying the executable objects based on the identified third category. (Lee, [0404]-[0405], “ According to an exemplary embodiment, the electronic device 100 may display a list of applications installed thereon, and receive an input of selecting at least one application in the application list. According to an exemplary embodiment, the electronic device 100 may identify the first object by analyzing the image of the first object and recommend at least one application corresponding to the first object. The electronic device 100 may receive an input of selecting the recommended application. ” and [0358]-[0361], “ According to an exemplary embodiment, object recognition model information may be generated by the electronic device 100 or a server. ... FIG. 26 is a sequence diagram illustrating a method of a server generating object recognition model information according to an exemplary embodiment. In operation S2610, a server 2600 may determine a plurality of categories and a plurality of keywords. For example, referring to FIG. 27, the server 2600 may determine product groups (e.g., a TV, an air conditioner, a vacuum cleaner, etc.) as categories 2710. ” and [0357], “ Meanwhile, according to an exemplary embodiment, object recognition model information may include information on applications (or controllers) linked to each of a plurality of objects. For example, object recognition model information may include information on a third application (e.g., a music player application) linked to a third object (e.g., a speaker). ” and [0364], “ According to an exemplary embodiment, the server 2600 may search for an application or a controller corresponding to a category and a keyword. For example, when the category 2710 is “TV” and the keyword 2720 is “living room,” the server 2600 may search a website for a controller corresponding to a living room TV. Also, when the category 2710 is “water bottle” and the keyword 2720 is “smart,” the server 2600 may search a website for an application of a smart water bottle. ” and [0370], “ According to an exemplary embodiment, the server 2600 may generate object recognition model information by matching each of the plurality of object recognition models and application (or controller) information with each other. For example, the object recognition model information generated by the server 2600 may include first matching information obtained by matching an object recognition model of the living room TV with controller information for controlling the living room TV, second matching information obtained by matching an object recognition model of a refrigerator with refrigerator management application information, and so on. ”; Examiner’s note: Examples disclosed by Lee above include executable applications for speakers [0357], TVs [0364], and refrigerators [0370] which are three different categories of devices.) 25. Claim 15 is similar in scope to claims 6, 11, and 12 except for an additional limitation that Lee in view of Kim discloses: [[The method of claim 12, wherein the displaying the executable objects comprises based on a second input indicating selection of one object among the executable objects, requesting, to the first external electronic device and the second external electronic device, execution of a function mapped to the object selected by the second input,]] and wherein the second input is different from the first input. (Lee, [0122]-[0123], “ For example, the user input for selecting the first object and the second object may be varied. For example, the user input for selecting the first object and the second object may be at least one of a touch input, a voice input, an ocular input, and a bending input, but is not limited thereto. ... For example, a touch input stated in an exemplary embodiment may be a tap, a touch and hold, a double tap, a drag, panning, a flick, a drag and drop, or so on. ” and [0133], “ Also, the electronic device 100 may receive an input of tapping, double tapping, or touching and holding each of the first object and the second object, but the touch input is not limited thereto. ” and [0149], “ An ocular input denotes an input of the user adjusting eye blinks, a gaze position, a moving speed of his or her eyeballs, etc. to control the electronic device 100. ” and [0134], “ According to an exemplary embodiment, the electronic device 100 may analyze the voice of the user and recognize identification information of the first object and identification information of the second object included in the voice of the user. ” and [0213], “ When the user wants to view a movie played on the TV 1001 in a dark environment, the electronic device 100 may receive a user input for selecting the TV 1001 and the light fixture 1002 through the live view. ”; Examiner’s note: There are many different input methods disclosed by Lee in [0122]-[0123]. For instance, a user could utilize two different input methods to select a TV and a light fixture. The user could use “touch and hold” to select the TV and a voice or ocular input to select the light fixture.) 26. As per claim 17, Lee in view of Kim discloses: The wearable device of claim 16, wherein the wearable device further comprises communication circuitry, and wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (See rejection for claim 16.) receive, through the communication circuitry, metadata with respect to the one or more first external electronic devices; (Lee, [0087], “ According to an exemplary embodiment, when three or more objects are recognized through the camera or a communication interface, the electronic device 100 may display the recognized objects on the screen and receive an input for selecting a first object and a second object among the recognized objects from the user. ” and Lee, [0177], “ The electronic device 100 may establish the first communication link and the second communication link by exchanging identification information (e.g., device IDs, media access control (MAC) addresses, device names, etc.), function information (e.g., support for BLE, Bluetooth, Ant+, Wi-Fi, and NFC), information on a preferred communication method (e.g., Bluetooth), etc. with the first object 801 and the second object 802. ” and Kim, [0069], “ According to an embodiment, the electronic device 100 may identify a type of the target device in order to acquire the virtual object data from the three-dimensional image data of the target device. Here, the type of the target device may be information for identifying a category to which the target device belongs. The type of the device according to an embodiment of the disclosure may include a product group, a model name, etc. of the target device. For example, when the target device is a refrigerator, the type may include ‘electronic device’, ‘home appliance’, ‘refrigerator’, ‘2-door refrigerator’, ‘IRS82M6000’, etc. The electronic device 100 may use geometric data, attribute data, metadata, file name, etc. of the target device included in the three-dimensional image data, in order to identify the type of the target device. ”) identify, based on the received metadata, the one or more first functions. (Lee, [0177], “ The electronic device 100 may establish the first communication link and the second communication link by exchanging identification information (e.g., device IDs, media access control (MAC) addresses, device names, etc.), function information (e.g., support for BLE, Bluetooth, Ant+, Wi-Fi, and NFC), information on a preferred communication method (e.g., Bluetooth), etc. with the first object 801 and the second object 802. ” and Lee, [0179], “ In operation S830, the first object 801 may transmit first attribute information to the electronic device 100. For example, the first object 801 may transmit the first attribute information (e.g., current state information of the first object 801, information on a function supported by the first object 801, etc.) to the electronic device 100 through the first communication link. ” and Lee, [0515]-[0516], “ The display 111 may display at least one attribute of a first object and a second object. For example, the display 111 may display an icon, a sign, or text corresponding to the at least one attribute. The display 111 may display a recommended operation list including a plurality of operations determined based on an attribute of the first object and an attribute of the second object. Also, the display 111 may display state information corresponding to the plurality of objects. ”) The motivation for this modification is the same as claim 2. 27. As per claim 18, Lee in view of Kim discloses: The wearable device of claim 17, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to : (See rejection for claim 17.) based on the received metadata, identify a category including the one or more first external electronic devices; and (Lee, [0179], “ In operation S830, the first object 801 may transmit first attribute information to the electronic device 100. For example, the first object 801 may transmit the first attribute information (e.g., current state information of the first object 801, information on a function supported by the first object 801, etc.) to the electronic device 100 through the first communication link. ” and Lee, [0177], “ The electronic device 100 may establish the first communication link and the second communication link by exchanging identification information (e.g., device IDs, media access control (MAC) addresses, device names, etc.), function information (e.g., support for BLE, Bluetooth, Ant+, Wi-Fi, and NFC), information on a preferred communication method (e.g., Bluetooth), etc. with the first object 801 and the second object 802. ” and Lee, [0553], “ The communication interface 2610 may include one or more components which enable communication between the server 2600 and the electronic device 100. For example, the communication interface 2610 may receive an image of an object, information on an application (a controller, a memo, or a website address) linked to the object, identification information of the object, and a request for generation of object recognition model information from the electronic device 100. ” and Lee, [0356], “ A model for recognizing an object (which will be referred to as an “object recognition model” below) may include images for identifying the object. For example, a first object recognition model for recognizing a first object (e.g., a TV) may include first images (e.g., TV images) corresponding to the first object, and a second object recognition model for recognizing a second object (e.g., a refrigerator) may include second images (e.g., refrigerator images) corresponding to the second object. ” and Lee, [0358]-[0361], “ According to an exemplary embodiment, object recognition model information may be generated by the electronic device 100 or a server. ... FIG. 26 is a sequence diagram illustrating a method of a server generating object recognition model information according to an exemplary embodiment. In operation S2610, a server 2600 may determine a plurality of categories and a plurality of keywords. For example, referring to FIG. 27, the server 2600 may determine product groups (e.g., a TV, an air conditioner, a vacuum cleaner, etc.) as categories 2710. ” and Kim, [0069], “ According to an embodiment, the electronic device 100 may identify a type of the target device in order to acquire the virtual object data from the three-dimensional image data of the target device. Here, the type of the target device may be information for identifying a category to which the target device belongs. The type of the device according to an embodiment of the disclosure may include a product group, a model name, etc. of the target device. For example, when the target device is a refrigerator, the type may include ‘electronic device’, ‘home appliance’, ‘refrigerator’, ‘2-door refrigerator’, ‘IRS82M6000’, etc. The electronic device 100 may use geometric data, attribute data, metadata, file name, etc. of the target device included in the three-dimensional image data, in order to identify the type of the target device. ”; Examiner’s note: As disclosed by Lee in [0179], an object (device) may transmit attribute information to the “wearable” electronic device related to functions supported by the object. Information about the object (device) can also be sent to the server from the “wearable” electronic device, as disclosed in [0552]. In [0360]-[0361], a server performs a “object recognition model” process on objects (devices) based on input information provided and assigns categories to the objects.) based on the identified category, identify the one or more first functions. (Lee, [0553], “ The communication interface 2610 may include one or more components which enable communication between the server 2600 and the electronic device 100. For example, the communication interface 2610 may receive an image of an object, information on an application (a controller, a memo, or a website address) linked to the object, identification information of the object, and a request for generation of object recognition model information from the electronic device 100. ” and Lee, [0358]-[0361], “ According to an exemplary embodiment, object recognition model information may be generated by the electronic device 100 or a server. ... FIG. 26 is a sequence diagram illustrating a method of a server generating object recognition model information according to an exemplary embodiment. In operation S2610, a server 2600 may determine a plurality of categories and a plurality of keywords. For example, referring to FIG. 27, the server 2600 may determine product groups (e.g., a TV, an air conditioner, a vacuum cleaner, etc.) as categories 2710. ” and Lee, [0364], “ According to an exemplary embodiment, the server 2600 may search for an application or a controller corresponding to a category and a keyword. For example, when the category 2710 is “TV” and the keyword 2720 is “living room,” the server 2600 may search a website for a controller corresponding to a living room TV. ”; Examiner’s note: Based on the "object recognition model" process on the server disclosed by Lee [0358]-[0361], an object (device) is categorized and the server looks for an appropriate application or controller (function) corresponding to its category [0364].) The motivation for this modification is the same as claim 2 . 07-21-aia AIA 28. Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US-2017/0185276-A1, hereinafter "Lee") in view of Kim et al. (US-2020/0380771-A1, hereinafter "Kim"), and further in view of Choi (WO-2020/149655-A1) . 29. As per claim 5, Lee in view of Kim discloses: The wearable device of claim 2, further comprising communication circuitry, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to obtain, through the communication circuitry, (See rejection for claim 2.) [[another]] metadata indicating functions applicable to each of a plurality of external electronic devices, [[and wherein the another metadata comprises the first metadata and the second metadata.]] (Lee, [0370]-[0372], “ According to an exemplary embodiment, the server 2600 may generate object recognition model information by matching each of the plurality of object recognition models and application (or controller) information with each other. … In operation S2640, the server 2600 may transmit the object recognition model information to the electronic device 100. According to an exemplary embodiment, the server 2600 may the transmit object recognition model information to the electronic device 100 when a request is received from the electronic device 100, or may transmit the object recognition model information to the electronic device 100 at regular time intervals. ” and Lee, [0356], “ A model for recognizing an object (which will be referred to as an “object recognition model” below) may include images for identifying the object. For example, a first object recognition model for recognizing a first object (e.g., a TV) may include first images (e.g., TV images) corresponding to the first object, and a second object recognition model for recognizing a second object (e.g., a refrigerator) may include second images (e.g., refrigerator images) corresponding to the second object. ” and Lee, [0357]-[0358], “ Meanwhile, according to an exemplary embodiment, object recognition model information may include information on applications (or controllers) linked to each of a plurality of objects. … According to an exemplary embodiment, object recognition model information may be generated by the electronic device 100 or a server. ” and Kim, [0069], “ According to an embodiment, the electronic device 100 may identify a type of the target device in order to acquire the virtual object data from the three-dimensional image data of the target device. Here, the type of the target device may be information for identifying a category to which the target device belongs. The type of the device according to an embodiment of the disclosure may include a product group, a model name, etc. of the target device. For example, when the target device is a refrigerator, the type may include ‘electronic device’, ‘home appliance’, ‘refrigerator’, ‘2-door refrigerator’, ‘IRS82M6000’, etc. The electronic device 100 may use geometric data, attribute data, metadata, file name, etc. of the target device included in the three-dimensional image data, in order to identify the type of the target device. ”; Examiner’s note: As disclosed by Lee in [0358], a server can utilize a “object recognition model” to identify a plurality of objects (devices) and determine their applications [0357]. In [0370]-[0372], after the generates an “object recognition model” for all recognized objects (devices), the server can transmit this “metadata” of all recognized objects to the “wearable” electronic device.) 30. Lee in view of Kim doesn't explicitly disclose but Choi discloses: [[The wearable device of claim 2, further comprising communication circuitry, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to obtain, through the communication circuitry,]] another [[metadata indicating functions applicable to each of a plurality of external electronic devices,]] and wherein the another metadata comprises the first metadata and the second metadata. (Choi, p. 13, [0081], “ According to one embodiment, the server (30) can transmit different metadata to electronic devices associated with the server (30) (e.g., a first electronic device (10) and a second electronic device (20)) depending on the attributes of the electronic device. For example, in operation 410, the server (30) can generate metadata based on the device attributes of the first electronic device (10) and metadata based on the device attributes of the second electronic device (20).) ” and p. 14-15, [0089], “ According to various embodiments, the server (30) may generate additional information based on the received shared information. According to one embodiment, the server (30) can generate additional information (e.g., data associated with the shared information or a data access address for accessing data associated with the shared information) based on the shared information and the device attributes of the electronic device (e.g., the second electronic device (20)) to share the shared information. For example, the server (30) can transmit additional information along with metadata and shared information to other electronic devices. ” and p. 12, [0073], “ According to various embodiments, the server (30) can generate secondary information associated with the received shared information. In operation 410, the server (30) can generate metadata (e.g., secondary information) based on shared information and/or device attribute information. ” and p. 12, [0076], “ According to one embodiment, the server (30) can generate metadata based on shared information and device attributes. For example, the server (30) can classify the type and/or category of content corresponding to the shared information according to the attributes of the device. ” and p. 12, [0077], “ According to various embodiments, the server (30) can transmit shared information and metadata to electronic devices belonging to the same account as the first electronic device (10). According to one embodiment, in operation 415, the server (30) can transmit shared information and metadata to the second electronic device (20). ” and p. 14, [0086], “ According to one embodiment, an electronic device (e.g., a first electronic device (10) and/or a second electronic device (20)) may provide shared information to a user based further on external electronic device recognition. ... As another example, an electronic device can recognize an external electronic device based on GPS information, indoor positioning, and/or image recognition. According to one embodiment, the electronic device can provide shared information based on the device properties of an external electronic device located in close proximity. ” and p. 11, [0068], “ According to various embodiments, device attributes can be classified based on the use of the electronic device. … As another example, the TV (301) may be used for multimedia playback. ” and p. 26, [0162], “ According to one embodiment, the TV (301) can transmit information about a designated application (e.g., a meal planner), information about an identified dish, and information about a device to be shared to the server (30) as shared information. The server (30) can transmit the received shared information to the refrigerator (304). ”; Examiner’s note: As disclosed by Choi in [0081], can generate a "first" metadata depending on the attributes of an electronic device. Further, in [0073], the server can also generate a "second" metadata (secondary information) based on shared information and/or device attributes. As indicated in [0089], "the server can transmit additional information along with metadata and shared information to other electronic devices" (the additional information based on the shared information).) 31. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the wearable device of claim 1 of Lee in view of Kim to include the disclosure of generating additional metadata describing electronic devices, of Choi. The motivation for this modification could have been to utilize additional metadata as a means to either update or provide further information about an electronic device. The additional metadata could be sent when there is a request for a specific electronic device function (and possibly seldom used by a user) rather than sending all functions in a single transfer. This could potentially save communication traffic between devices for only when needed . 07-21-aia AIA 32. Claim s 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al. (US-2017/0185276-A1, hereinafter "Lee") in view of Kim et al. (US-2020/0380771-A1, hereinafter "Kim"), and further in view of Okubo et al. (US-2016/0373269-A1, hereinafter "Okubo") . 33. As per claim 9, Lee in view of Kim discloses: The wearable device of claim 1, further comprising: (See rejection for claim 1.) a sensor to track an eye gaze of a user wearing the wearable device; and (Lee, [0149], “ An ocular input denotes an input of the user adjusting eye blinks, a gaze position, a moving speed of his or her eyeballs, etc. to control the electronic device 100. For example, the electronic device 100 may sense a user input of looking at the TV for three seconds or more on the live view, moving his or her eyes to the light fixture, and then looking at the light fixture for three seconds or more. The electronic device 100 may analyze the gaze positions of the user and determine that the user has selected the TV and the light fixture. ” and [0538], “ The eyeball tracking module 152 may analyze eye blinks, a gaze position, and/or a moving speed of an eyeball to interpret an ocular input of the user. ” and [0065], “ The electronic device 100 described herein may a device wearable by a user. ”) a microphone to receive an audio signal, (Lee, [0530], “ The microphone 142 receives and processes an external sound signal into electrical voice data. For example, the microphone 142 may receive a sound signal from an external device or a speaker. ”) wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to: (See rejection for claim 1.) [[identify a preset keyword]] from the audio signal received by the microphone; and (Lee, [0530], “ The microphone 142 receives and processes an external sound signal into electrical voice data. For example, the microphone 142 may receive a sound signal from an external device or a speaker. ” and [0134], “ According to an exemplary embodiment, the electronic device 100 may analyze the voice of the user and recognize identification information of the first object and identification information of the second object included in the voice of the user. ”) [[based on identifying the preset keyword,]] receive the input based on the eye gaze detected by the sensor and a time section [[in which the preset keyword is identified.]] (Lee, [0149], “ An ocular input denotes an input of the user adjusting eye blinks, a gaze position, a moving speed of his or her eyeballs, etc. to control the electronic device 100. For example, the electronic device 100 may sense a user input of looking at the TV for three seconds or more on the live view, moving his or her eyes to the light fixture, and then looking at the light fixture for three seconds or more. The electronic device 100 may analyze the gaze positions of the user and determine that the user has selected the TV and the light fixture. ” and [0122]-[0123], “ For example, the user input for selecting the first object and the second object may be varied. For example, the user input for selecting the first object and the second object may be at least one of a touch input, a voice input, an ocular input, and a bending input, but is not limited thereto. ... For example, a touch input stated in an exemplary embodiment may be a tap, a touch and hold, a double tap, a drag, panning, a flick, a drag and drop, or so on. ”) 34. Lee in view of Kim doesn't explicitly disclose but Okubo discloses: identify a preset keyword [[from the audio signal received by the microphone; and]] (Okubo, [0135]- [0136], “ For example, in a case where the user says “Turn on living room light”, keyword matching of “living room”, “lighting”, and “turn on” fill in all frames (control command (power_on), device (lighting), and location (living room)) as action information. Accordingly, processing is performed to turn on the lighting installed in the living room. Also, in a case where the user says “Turn on air conditioner”, keyword matching of “air conditioner” and “turn on” fill in the frames other than location (control command (power_on) and device (air conditioner))) as action information. ” and [0054], “ As for a method to identify the electronic device that the user wants to confirm or operate, and the contents of confirmation operation, there is what is called slot filling technology, where the speech of the user is analyzed, and the results are fit into a frame (template) to advance the conversation. For example, a frame is made up of three types of information, that are “device”, “operation”, and “location”. ”) based on identifying the preset keyword, [[receive the input based on the eye gaze detected by the sensor and a time section]] in which the preset keyword is identified. (Okubo Figs. 6, 7, 12, 18; [0135]- [0136], “ For example, in a case where the user says “Turn on living room light”, keyword matching of “living room”, “lighting”, and “turn on” fill in all frames (control command (power_on), device (lighting), and location (living room)) as action information. Accordingly, processing is performed to turn on the lighting installed in the living room. Also, in a case where the user says “Turn on air conditioner”, keyword matching of “air conditioner” and “turn on” fill in the frames other than location (control command (power_on) and device (air conditioner))) as action information. ” and [0173]-[0174], “ FIG. 14 is a diagram illustrating an example of operations of the voice recognition processing (S321 in FIG. 12). First, the home gateway 1102 instructs the sound collection device 2013 to store the sound (S351), and instructs the voice conversation server 2100 to start voice recognition (S352). The sound collection device 2013 that has received this instruction starts saving the sound data that the user has spoken (S353). ” and [0133], “ The speech information is set at the time of the speech start trigger (line-of-sight) of the user being detected by the line-of-sight detecting unit 2111, line-of-sight detecting unit 2121, or line-of-sight detecting unit 2131, of the first device 2011, second device 2012, or sound collection device 2013. ”; Examiner’s note: Figs. 6, 7, 12, 18 each show a “timeline” of events for the voice “conversation” recognition process.) 35. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the wearable device of claim 1 of Lee in view of Kim to include the disclosure utilizing voice command keywords to select and control electronic devices in a given timespan, of Okubo. The motivation for this modification could have been to assist users to easily “select” electronic devices by just mentioning the device, its function, and location. This provides for a hands-free interface so that a user can easily adjust things like lighting, air temperature, and changing a channel on a TV. 36. As per claim 10, Lee in view of Kim, and further in view of Okubo discloses: The wearable device of claim 9, wherein the instructions, when executed by at least one processor individually or collectively, cause the wearable device to, (See rejection for claim 9.) based on identifying another keyword received after the preset keyword from the audio signal, request, to the first external electronic device and the second external electronic device, execution of a function indicated by the another keyword. (Lee, [0134], “ According to an exemplary embodiment, the electronic device 100 may analyze the voice of the user and recognize identification information of the first object and identification information of the second object included in the voice of the user. ” and Okubo, [0136], “ Also, in a case where the user says “Turn on air conditioner”, keyword matching of “air conditioner” and “turn on” fill in the frames other than location (control command (power_on) and device (air conditioner))) as action information. If there is only one air conditioner in the house, processing to turn the power of that air conditioner on is performed. If there are two or more air conditioners installed, the user is queried for clarification by a question “Where is the air conditioner to turn on?” to which the user specifies the location. This, the air conditioner for which the power should be turned on is finally identified. ” and Okubo, [0228], “ In the example illustrated in FIG. 20A, the first device 2011 is the air conditioner installed in the living room, the second device 2012 is the air conditioner installed in the bedroom, and the sound collection device 2013 is the microphone installed on the ceiling. In a case where the user says “Warm it up” without directing line-of-sight toward the living room devices, the living room air conditioner that is installed in the same room as the user is selected as the object device. In a case where multiple air conditioners are installed in the living room, all of the air conditioners may be selected, of just the closest device to the user may be selected. ” and Okubo, [0138], “ Also, there are cases, such as the user saying “All off”, where operation can be performed even if only the control command (all_off) of the action information is filled in. ”; Examiner’s note: As disclosed by Okubo in [0136], multiple keywords are used in a conversation to clarify the action of turning on an air conditioner, such as if the action was directed toward a first air conditioner or a second air conditioner. In Okubo [228], multiple devices, in this case air conditioners, may be all selected simultaneously so that all selected devices are executed. Lastly, in Okubo [138], there is a scenario to turn off all electronic devices off with the combination of keywords “All” and “off.”) 37. Before the effective filing date of the claimed invention, it would have been obvious to a person having ordinary skill in the art to modify the wearable device of claim 9 of Lee in view of Kim to include the disclosure utilizing additional voice command keywords to select and control electronic devices, of Okubo. The motivation for this modification could have been to allow a user to potentially further clarify a voice command or command multiple electronic devices at once. By using multiple keywords, a user could more quickly issue a command to multiple devices rather than speak commands one at a time . Conclusion 07-96 AIA 38. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kaehler et al. (US-2022/0083198-A1) discloses a wearable device that can recognize external electronic devices and interact with them with a virtual remote . 39. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW CLOTHIER whose telephone number is (571)272-4667. The examiner can normally be reached Mon-Fri 8:00am-4:00pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /MATTHEW CLOTHIER/Examiner, Art Unit 2614 /KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614 Application/Control Number: 18/962,983 Page 2 Art Unit: 2614 Application/Control Number: 18/962,983 Page 3 Art Unit: 2614 Application/Control Number: 18/962,983 Page 4 Art Unit: 2614 Application/Control Number: 18/962,983 Page 5 Art Unit: 2614 Application/Control Number: 18/962,983 Page 6 Art Unit: 2614 Application/Control Number: 18/962,983 Page 7 Art Unit: 2614 Application/Control Number: 18/962,983 Page 8 Art Unit: 2614 Application/Control Number: 18/962,983 Page 9 Art Unit: 2614 Application/Control Number: 18/962,983 Page 10 Art Unit: 2614 Application/Control Number: 18/962,983 Page 11 Art Unit: 2614 Application/Control Number: 18/962,983 Page 12 Art Unit: 2614 Application/Control Number: 18/962,983 Page 13 Art Unit: 2614 Application/Control Number: 18/962,983 Page 14 Art Unit: 2614 Application/Control Number: 18/962,983 Page 15 Art Unit: 2614 Application/Control Number: 18/962,983 Page 16 Art Unit: 2614 Application/Control Number: 18/962,983 Page 17 Art Unit: 2614 Application/Control Number: 18/962,983 Page 18 Art Unit: 2614 Application/Control Number: 18/962,983 Page 19 Art Unit: 2614 Application/Control Number: 18/962,983 Page 20 Art Unit: 2614 Application/Control Number: 18/962,983 Page 21 Art Unit: 2614 Application/Control Number: 18/962,983 Page 22 Art Unit: 2614 Application/Control Number: 18/962,983 Page 23 Art Unit: 2614 Application/Control Number: 18/962,983 Page 24 Art Unit: 2614 Application/Control Number: 18/962,983 Page 25 Art Unit: 2614 Application/Control Number: 18/962,983 Page 26 Art Unit: 2614 Application/Control Number: 18/962,983 Page 27 Art Unit: 2614 Application/Control Number: 18/962,983 Page 28 Art Unit: 2614 Application/Control Number: 18/962,983 Page 29 Art Unit: 2614 Application/Control Number: 18/962,983 Page 30 Art Unit: 2614 Application/Control Number: 18/962,983 Page 31 Art Unit: 2614 Application/Control Number: 18/962,983 Page 32 Art Unit: 2614 Application/Control Number: 18/962,983 Page 33 Art Unit: 2614 Application/Control Number: 18/962,983 Page 34 Art Unit: 2614 Application/Control Number: 18/962,983 Page 35 Art Unit: 2614 Application/Control Number: 18/962,983 Page 36 Art Unit: 2614 Application/Control Number: 18/962,983 Page 37 Art Unit: 2614 Application/Control Number: 18/962,983 Page 38 Art Unit: 2614 Application/Control Number: 18/962,983 Page 39 Art Unit: 2614 Application/Control Number: 18/962,983 Page 40 Art Unit: 2614 Application/Control Number: 18/962,983 Page 41 Art Unit: 2614 Application/Control Number: 18/962,983 Page 42 Art Unit: 2614 Application/Control Number: 18/962,983 Page 43 Art Unit: 2614
Prosecution Insights