Detailed Action
Notice of Pre-AIA or AIA status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
At the time of this writing, the Applicant has filed at least 30 information disclosure statements in this application, citing thousands of references over hundreds of pages. In order to give each filing sufficient attention, the Examiner will review the information disclosure statements in batches with each Office Action, and review all/any remaining information disclosure statements if/when the application reaches condition for allowance.
The information disclosure statements filed on March 5, 2024, March 17, 2025, March 28, 2025, April 8, 2025, and April 22, 2025 comply with the provisions of 37 C.F.R. § 1.97, 1.98, and MPEP § 609, and therefore have been placed in the application file. The information referred to therein has been considered as to the merits.
Any remaining information disclosure statement not explicitly mentioned above has not yet been reviewed, but will be reviewed in due course.
Claim Rejections – 35 U.S.C. § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
I. Su discloses claims 112, 113, 118–121, 125, and 128–130.
Claims 112, 113, 118–121, 125, and 128–130 are rejected under 35 U.S.C. § 102(a)(2) as being anticipated by U.S. Patent Application Publication No. 2023/0333803 A1 (“Su”).
Claim 112
Su discloses:
A first computer system configured to communicate with a display generation component and one or more input devices, comprising: one or more processors; and memory storing one or more programs configured to be executed by the one or more processors,
“FIG. 2 is a schematic diagram of a structure of a mobile phone according to an embodiment of this disclosure. The mobile phone may be the foregoing mobile phone 100, and may alternatively be the mobile phone 200.” Su ¶ 68. In either case, “[t]he mobile phone may include a processor 110, an external memory interface 120, an internal memory 121,” “a display screen 194,” and “a touch sensor 180K,” among other things. Su ¶ 69.
the one or more programs including instructions for:
Reference is made below to FIGS. 4A–4C, which illustrates two mobile phones 100 and 200 each demonstrating different functions performed by the same instructions illustrated in the software stack shown in FIG. 3. See Su ¶¶ 92, 117, and 120. Of particular note is “video call application A2,” which comprises instructions that cause either mobile phone 100 or 200 to perform the functions described below.
detecting, via the one or more input devices, a request to display an application that uses data captured by a camera sensor;
Su discloses that video call application A2 provides for at least two ways to request that video call application A2 be displayed. A first path for requesting display of the video call application A2 occurs “[i]f the mobile phone 100 detects a single-tap operation performed by a user U1 on the video call application A2 on the mobile phone.” Su ¶ 121.
Another path for requesting display of the video call application A2 involves “detecting a single-tap operation performed by the user U2 on an ‘accept’ control” shown in FIG. 4B, e.g., in a case where there is an incoming video call. Su ¶ 123.
and in response to detecting the request to display the application that uses data captured by a camera sensor:
in accordance with a determination that the first computer system is connected to a second computer system that is in communication with one or more camera sensors, displaying, via the display generation component, the application;
“If detecting a single-tap operation performed by the user U2 on an ‘accept’ control, the mobile phone 200 sends a video call setup response to the mobile phone 100,” Su ¶ 123, and as a result, “the mobile phone 100 and the mobile phone 200 each display a video call interface.” Su ¶ 124.
and in accordance with a determination that the first computer system is not connected to a computer system that is in communication with one or more camera sensors, displaying, via the display generation component, a first connection user interface element that, when selected, initiates a process for connecting the first computer system with the second computer system.
On the other hand, “[i]f the mobile phone 100 detects a single-tap operation performed by a user U1 on the video call application A2 on the mobile phone 100, the video call application A2 is opened on the mobile phone 100, and the mobile phone 200 displays a contact selection interface shown in FIG. 4B.” Su ¶ 121. Here, additional steps must be performed (i.e., the beginning of a process) for connecting the mobile phone 100 with mobile phone 200. See Su ¶¶ 122–123.
Claim 113
Su discloses the first computer system of claim 112, the one or more programs further including instructions for:
in response to detecting the request to display the application that uses data captured by a camera sensor and in accordance with the determination that the first computer system is not connected to a computer system that is in communication with one or more camera sensors, displaying, via the display generation component, a list of one or more connection user interface elements for connecting the first computer system with respective computer systems,
“If the mobile phone 100 detects a single-tap operation performed by a user U1 on the video call application A2 on the mobile phone 100, the video call application A2 is opened on the mobile phone 100, and the mobile phone 200 displays a contact selection interface shown in FIG. 4B.” Su ¶ 121.
the list of one or more connection user interface elements including the first connection user interface element.
The contact selection interface, as shown in FIG. 4B, includes a list of contacts, each one having a respective “video call control 10” for initiating a video call request to the corresponding contact. See Su ¶ 122.
Claim 118
Su discloses the first computer system of claim 112, the one or more programs further including instructions for:
in response to detecting the request to display the application that uses data captured by a camera sensor and in accordance with the determination that the first computer system is not connected to a computer system that is in communication with one or more camera sensors, displaying, via the display generation component, instructions to select a user associated with a computer system that includes one or more camera sensors that are configured for use with the first computer system.
“If the mobile phone 100 detects a single-tap operation performed by a user U1 on the video call application A2 on the mobile phone 100, the video call application A2 is opened on the mobile phone 100, and the mobile phone 200 displays a contact selection interface shown in FIG. 4B.” Su ¶ 121. The interface itself, with all of its icons, falls within the scope of “instructions to select a user associated with a computer system that includes one or more camera sensors that are configured for use with the first computer system,” because differences between the underlying meaning of a human readable message recited in a claim and the underlying meaning of a human readable message described in a prior art reference are not patentable. MPEP § 2111.05.
Claim 119
Su discloses the first computer system of claim 112, the one or more programs further including instructions for:
detecting, via the one or more input devices, a selection of the first connection user interface element;
Mobile phone 100 is configured to “detect[] a single-tap operation performed by the user U1 on a video call control 10 corresponding to a ‘user U2’ in the contact list.” Su ¶ 122.
and after detecting selection of the first connection user interface element, displaying, via the display generation component, instructions to confirm on the second computer system a request to connect the second computer system with the first computer system.
“If detecting a single-tap operation performed by the user U1 on a video call control 10 corresponding to a ‘user U2’ in the contact list, . . . mobile phone 100 sends a video call request to the mobile phone 200, a device corresponding to the user U2, to request to set up a video call,” Su ¶ 122, and responsive to this request, “the mobile phone 200 displays prompt information 20,” asking U2 whether he/she wishes to accept or cancel the call. Su ¶ 123.
Claim 120
Su discloses the first computer system of claim 112, the one or more programs further including instructions for:
receiving an indication that a request to connect the second computer system with the first computer system has been accepted;
After sending a video call request to mobile phone 200, and after user U2 of mobile phone 200 selects the “accept” control, “the mobile phone 200 sends a video call setup response to the mobile phone 100.” Su ¶ 123.
and in response to receiving the indication that the request to connect the second computer system with the first computer system has been accepted, displaying, via the display generation component, a representation of video captured by one or more camera sensors of the second computer system.
“The mobile phone 100 usually displays, by default, a video call image of the user U1 (for example, a video call image shown in a female avatar) in an upper right corner of the screen, and displays a video call image of the user U2 (for example, a video call image shown in a male avatar) in a main area of the screen.” Su ¶ 125.
Claim 121
Su discloses the first computer system of claim 112, the one or more programs further including instructions for:
receiving an indication that the second computer system is connected to the first computer system;
“After the mobile phone 100 receives [a] video call setup response sent by the mobile phone 200, a video call is set up between the mobile phone 100 and the mobile phone 200.” Su ¶ 124.
and in response to receiving the indication that the second computer system is connected to the first computer system, displaying, via the display generation component, a representation of video captured by one or more camera sensors of the second computer system.
“In addition [to setting up the call], the mobile phone 100 and the mobile phone 200 each display a video call interface,” Su ¶ 124, which includes the image captured by the camera. Su ¶ 125.
Claim 125
Su discloses the first computer system of claim 112, wherein
in response to receiving an indication that the second computer system has been selected, the second computer system displays an accept option for connecting the second computer system with the first computer system.
“If detecting a single-tap operation performed by the user U1 on a video call control 10 corresponding to a ‘user U2’ in the contact list, . . . mobile phone 100 sends a video call request to the mobile phone 200, a device corresponding to the user U2, to request to set up a video call,” Su ¶ 122, and responsive to this request, “the mobile phone 200 displays prompt information 20,” asking U2 whether he/she wishes to accept or cancel the call. Su ¶ 123.
Claim 128
Su discloses the first computer system of claim 125,
wherein the second computer system detects a selection of the accept option for connecting the second computer system with the first computer system
“If detecting a single-tap operation performed by the user U2 on an ‘accept’ control, the mobile phone 200 sends a video call setup response to the mobile phone 100.” Su ¶ 123.
and, in response to the second computer system detecting the selection of the accept option for connecting the second computer system with the first computer system, the second computer system initiates a process for connecting with the first computer system.
“Refer to FIG. 4C. After the mobile phone 100 receives the video call setup response sent by the mobile phone 200, a video call is set up between the mobile phone 100 and the mobile phone 200. In addition, the mobile phone 100 and the mobile phone 200 each display a video call interface.” Su ¶ 124.
Claim 129
Claim 129 recites a non-transitory computer readable storage medium that stores exactly the same programs as are stored on the memory of the system recited in claim 112. Since Su discloses the memory in the system of claim 112 and all of its stored programs, and since a memory is but one example of a non-transitory computer readable storage medium (i.e., a memory is a species of the genus of computer readable media), claim 129 is rejected over all of the findings set forth in the rejection of claim 112. See MPEP § 2131.02 (subsection II.).
Claim 130
Claim 130 recites the exact same method that the system of claim 112 performs as part of its normal operation. “Under the principles of inherency, if a prior art device, in its normal and usual operation, would necessarily perform the method claimed, then the method claimed will be considered to be anticipated by the prior art device.” MPEP § 2112.02. Therefore, claim 130 is rejected over all of the findings and rationale as provided above for claim 112.
II. Carrigan discloses at least claims 112, 121–123, 129, and 130.
Claims 112, 121–123, 129, and 130 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by U.S. Patent Application Publication No. 2019/0342621 A1 (“Carrigan”).
Claim 112
Carrigan discloses:
A first computer system configured to communicate with a display generation component and one or more input devices, comprising: one or more processors; and memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
“FIG. 6A illustrates electronic device 600 with touch-sensitive display 602. In some embodiments, device 600 includes some or all of the features of devices 100, 300, and 500,” Carrigan ¶ 267, and those devices 100, 300, and 500 include, among other things, “memory 102 (which optionally includes one or more computer-readable storage mediums), memory controller 122, one or more processing units (CPUs) 120, peripherals interface 118, RF circuitry 108, audio circuitry 110, speaker 111, microphone 113, input/output (I/O) subsystem 106, other input control devices 116, and external port 124.” Carrigan ¶ 114.
detecting, via the one or more input devices, a request to display an application
Device 600 detects that was “physically rotated from the portrait orientation in FIG. 10A to the landscape orientation in FIG. 10B,” which the device interprets as a request to display user interface 1000 “in a landscape orientation.” Carrigan ¶ 326. Such rotations are detected via “input mechanism 508,” which includes an accelerometer 534 and/or gyroscope 536. Carrigan ¶ 251.
that uses data captured by a camera sensor;
Furthermore, user interface 1000 is “focused on the sources of video data (e.g., Cameras 1-6).” Carrigan ¶ 326.
and in response to detecting the request to display the application that uses data captured by a camera sensor: in accordance with a determination that the first computer system is connected to a second computer system that is in communication with one or more camera sensors, displaying, via the display generation component, the application;
“As illustrated in FIG. 10B, in accordance with a determination that device 600 is oriented in a landscape orientation (e.g., in response to device 600 being physically rotated from the portrait orientation in FIG. 10A to the landscape orientation in FIG. 10B), device 600 displays user interface 1000 focused on the sources of video data (e.g., Cameras 1-6).” Carrigan ¶ 326. Importantly, the display of user interface 1000 is created in accordance with a determination that one or more “source[s] of video data . . . are set to operate in a ‘stream’ mode in which the source provides a live video feed.” Carrigan ¶ 328.
These “sources of video data” come from cameras, which themselves are in communication with “hub devices” such as an “AppleTV or iPad.” Carrigan ¶ 367.
and in accordance with a determination that the first computer system is not connected to a computer system that is in communication with one or more camera sensors, displaying, via the display generation component, a first connection user interface element that, when selected, initiates a process for connecting the first computer system with the second computer system.
Displaying user interface 1000 further requires a determination of which external source of video is not in communication with one or more cameras, because displaying user interface 1000 includes displaying camera representations 1010 that are modified depending on whether “the corresponding source of data is set to operate in an ‘off’ mode in which the corresponding source provides no live video feed” (e.g., camera representation 1010e) Carrigan ¶ 328.
All of the camera representations 1010 in this menu “can be selected to display a user interface dedicated to the source corresponding to the selected image,” Carrigan ¶ 337; see also Carrigan ¶ 353. Performing such a selection may be the beginning of a process for changing the settings of the corresponding video source, illustrated and outlined in FIGS. 12D–12H. In short, the process involves (1) providing a user input 1250c1 selecting the camera’s settings screen (Carrigan ¶ 354–355 referring to FIG. 12D), (2) providing another user input 1250d (e.g., a tap) corresponding to selection of affordance 1208 for displaying streaming and recording settings (Carrigan ¶¶ 356–357 referring to FIG. 12F), and finally, (3) providing a user input 1250e (e.g., a tap) corresponding to selection of affordance 1216b and operational mode “STREAM” for the first context, thereby setting the camera’s streaming settings from “OFF” to “STREAM” (Carrigan ¶¶ 359–360 (referring to FIGS. 12H–12I).
Claim 121
Carrigan discloses the first computer system of claim 112, the one or more programs further including instructions for:
receiving an indication that the second computer system is connected to the first computer system; and in response to receiving the indication that the second computer system is connected to the first computer system, displaying, via the display generation component, a representation of video captured by one or more camera sensors of the second computer system.
“As illustrated in FIG. 10B, user interface 1000 includes camera representations 1010a-1010f (e.g., images) for the sources of video data.” Carrigan ¶ 326.
Claim 122
Carrigan discloses the first computer system of claim 121, the one or more programs further including instructions for:
displaying, via the display generation component, a user interface of the application, including displaying, in the user interface of the application, one or more control user interface elements that, when selected, cause the first computer system to perform respective functions associated with the video captured by one or more camera sensors of the second computer system.
“FIG. 8A illustrates device 600 displaying user interface 800, which includes a live video feed from the front door camera (described above) and scrubber bar 620.” Carrigan ¶ 329. The scrubber bar “is an interactive, elongated region on the display (e.g., 602) that includes a representation of media content (e.g., 624a, 624b) that can be scrolled along the direction parallel to direction of elongation. In some embodiments, the media content (e.g., the video) can be played back at arbitrary and variable rates based on a characteristic (e.g., the speed of a received user input).” Carrigan ¶ 293.
Claim 123
Carrigan discloses the first computer system of claim 122,
wherein displaying the one or more control user interface elements includes displaying a first control user interface element, and the one or more programs further including instructions for:
In addition to the user interface elements mentioned in the rejection of claim 122, Carrigan further discloses that user interface 800 includes a “related devices affordance 636A for accessing controls for at least one controllable external device that is associated with the front door camera.” Carrigan ¶ 307.
detecting, via the one or more input devices, a selection of the first control user interface element;
“At FIG. 8A, device 600 receives (e.g., detects) user input 850a (e.g., a tap) corresponding to selection of related devices affordance 636A.” Carrigan ¶ 308.
and in response to detecting the selection of the first control user interface element, displaying, via the display generation component:
“As illustrated in FIG. 8B, in response to receiving user input 850a, device 600 displays user interface 802.” Carrigan ¶ 308.
an indication that the second computer system is actively connected with the first computer system;
Within user interface 802, we still see a partial live view of the video from the video source. Carrigan FIG. 8B.
a user interface element corresponding to the second computer system;
User interface 802 further includes affordance 804a, for changing the settings of the camera currently being viewed. Carrigan FIG. 8B.
an indication that other computer systems are available to connect with the first computer system;
“As illustrated in FIG. 8C, device 600 receives (e.g., detects) user input 850b (e.g., a tap) corresponding to selection of affordance 804e representing a garage door camera. In response to receiving user input 850b, device 600 initiates a process for controlling the selected controllable external device. As illustrated in FIG. 8D, device 600 ceases to display the live video feed from the front door camera and replaces the live video feed from the front door camera with separate user interface 806 having controls (e.g., scrubber bar 620, etc.) for the garage door camera.” Carrigan ¶ 309.
and a user interface element corresponding to a third computer system that is different from the first computer system and the second computer system.
Other affordances in the menu “represent respective controllable external devices.” Carrigan ¶ 308.
Claim 129
Claim 129 recites a non-transitory computer readable storage medium that stores exactly the same programs as are stored on the memory of the system recited in claim 112. Since Carrigan discloses the memory in the system of claim 112 and all of its stored programs, and since a memory is but one example of a non-transitory computer readable storage medium (i.e., a memory is a species of the genus of computer readable media), claim 129 is rejected over all of the findings set forth in the rejection of claim 112. See MPEP § 2131.02 (subsection II.).
Claim 130
Claim 130 recites the exact same method that the system of claim 112 performs as part of its normal operation. “Under the principles of inherency, if a prior art device, in its normal and usual operation, would necessarily perform the method claimed, then the method claimed will be considered to be anticipated by the prior art device.” MPEP § 2112.02. Therefore, claim 130 is rejected over all of the findings and rationale as provided above for claim 112.
Claim Rejections – 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were effectively filed absent any evidence to the contrary. Applicant is advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned at the time a later invention was effectively filed in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
I. Fitzpatrick and Kim teach claims 112–114, 116, 117, 121, 122, 129, and 130.
Claims 112–114, 116, 117, 121, 122, 129, and 130 are rejected under 35 U.S.C. § 103 as being unpatentable over U.S. Patent Application Publication No. 2022/0201052 (hereafter “Fitzpatrick”) in view of U.S. Patent Application Publication No. 2012/0198098 A1 (hereafter “Kim”).
Claim 112
Fitzpatrick teaches:
A first computer system configured to communicate with a display generation component and one or more input devices, comprising:
Referring to FIGs. 1 and 2, Fitzpatrick teaches a control device 5 that communicates with its own display unit 50 (or 51, in FIG. 2), and user interface components. See Fitzpatrick ¶¶ 146 and 171.
one or more processors; and memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
Control device 5 further includes a processor 53, and a memory 54 that executes programs 61–62 (among others), which will be discussed below. See Fitzpatrick ¶¶ 146–147.
detecting, via the one or more input devices, a request to display an application that uses data captured by a camera sensor;
“The control device 5 is configured to allow a user to download, via the telecommunication module 52, both a video conferencing program 62 and a video control program 61 to the memory where each can be simultaneously executed by the control device 5.” Fitzpatrick ¶ 147. This disclosure falls within the scope of the claimed “request,” because it is the user who is causing programs 61–62 to be downloaded and executed, rather than the applications automatically downloading and installing themselves. See, e.g., Fitzpatrick ¶ 144.
and in response to detecting the request to display the application that uses data captured by a camera sensor:
in accordance with a determination that the first computer system is connected to a second computer system that is in communication with one or more camera sensors, displaying, via the display generation component, the application;
In a case where program 61 has already been paired with capture device 10, “[a] video stream, captured and processed by the capture device 10 can be transmitted via the local connection 6 to the control device 5 for use in a video conference, as governed by the video conferencing program 62.” Fitzpatrick ¶ 162.
and in accordance with a determination that the first computer system is not connected to a computer system that is in communication with one or more camera sensors, displaying, via the display generation component, a first connection user interface
On the other hand, at a time when “device 5, 10 [are] initially unlinked to one another, . . . at least one of the capture device 10 and control device 5, configured by their respective app 21 or video control program 61, initiates a pairing routine with the other.” Fitzpatrick ¶ 151.
With respect to a first connection user interface element, Fitzpatrick at least discloses on-screen elements for both facilitating the pairing process, see Fitzpatrick ¶ 152, as well as for performing a key or code exchange between the devices that are to be paired. Fitzpatrick ¶ 154. That said, Fitzpatrick does not appear to explicitly disclose whether there such a user interface element might also provide a mechanism for manually triggering the pairing routine.
Kim, however, teaches a system configured to perform the step of:
in accordance with a determination that the first computer system is not connected to a computer system . . . displaying, via the display generation component, a first connection user interface element
In response to receiving a user input requesting discovery of yet-to-be connected devices, “connection input unit 100 of the device that requests the discover displays a list of connectable devices on the display 102 based on the received device discover response message as illustrated in FIG. 2.” Kim ¶ 32.
that, when selected, initiates a process for a first connection user interface element connecting the first computer system with the second computer system.
As shown in FIG. 6, in response to selecting a device from the list, a process comprising steps 621–643 is performed, which involves pairing the devices. Kim ¶ 47.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to improve Fitzpatrick’s video control program 61 by implementing the pairing routine to be user-initiated. Note that Fitzpatrick does not prescribe an automatic or manual mode of operation for the pairing; it leaves that implementation detail as a choice for the skilled artisan to make.
One of ordinary skill in the art would have been motivated to follow Kim’s guidance of providing a user-initiated trigger for discovering nearby devices (i.e., Fitzpatrick’s capture device(s) 10) and/or connecting to those discovered devices, because embodiments of Fitzpatrick’s arrangement call for both devices 5 and 10 to be battery powered, see Fitzpatrick ¶¶ 139, 149, and 150, necessitating a more frugal approach to energy consumption. To that end, a skilled artisan would have wanted to wait until the user is ready to proceed before engaging in the pairing process.
Claim 113
Fitzpatrick teaches the first computer system of claim 112, the one or more programs further including instructions for:
in response to detecting the request to display the application that uses data captured by a camera sensor and in accordance with the determination that the first computer system is not connected to a computer system that is in communication with one or more camera sensors, displaying, via the display generation component,
At this stage of the process, Fitzpatrick teaches displaying on-screen elements for both facilitating the pairing process, see Fitzpatrick ¶ 152, as well as for performing a key or code exchange between the devices that are to be paired. Fitzpatrick ¶ 154.
That said, as explained in the rejection of claim 112, Fitzpatrick does not explicitly disclose the claimed selectable first connection user interface element.
Kim, however, teaches:
in response to detecting the request to display the application that uses data captured by a camera sensor and in accordance with the determination that the first computer system is not connected to a computer system that is in communication with one or more camera sensors, displaying, via the display generation component, a list of one or more connection user interface elements for connecting the first computer system with respective computer systems, the list of one or more connection user interface elements including the first connection user interface element.
In response to receiving a user input requesting discovery of yet-to-be connected devices, “connection input unit 100 of the device that requests the discover displays a list of connectable devices on the display 102 based on the received device discover response message as illustrated in FIG. 2.” Kim ¶ 32. As shown in FIG. 6, in response to selecting a device from the list, a process comprising steps 621–643 is performed, which involves pairing the devices. Kim ¶ 47.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to improve Fitzpatrick’s video control program 61 by implementing the pairing routine to be user-initiated. Note that Fitzpatrick does not prescribe an automatic or manual mode of operation for the pairing; it leaves that implementation detail as a choice for the skilled artisan to make.
One of ordinary skill in the art would have been motivated to follow Kim’s guidance of providing a user-initiated trigger for discovering nearby devices (i.e., Fitzpatrick’s capture device(s) 10) and/or connecting to those discovered devices, because embodiments of Fitzpatrick’s arrangement call for both devices 5 and 10 to be battery powered, see Fitzpatrick ¶¶ 139, 149, and 150, necessitating a more frugal approach to energy consumption. To that end, a skilled artisan would have wanted to wait until the user is ready to proceed before engaging in the pairing process.
Claim 114
Fitzpatrick and Kim teach the first computer system of claim 113,
wherein the list of one or more connection user interface elements includes connection user interface elements that correspond to respective computer systems that are signed into a same account as the first computer system.
As an initial matter, the Examiner observes the Applicant’s intention to accord the word “account” with a very broad scope, by providing a special description of “signed into a same account” in the specification to include devices that are signed into the same “user account and/or Wi-Fi network.” (Spec. ¶ 256) (emphasis added). “Under a broadest reasonable interpretation (BRI), words of the claim must be given their plain meaning, unless such meaning is inconsistent with the specification.” MPEP § 2111.01. Therefore, the broadest reasonable interpretation of “respective computer systems that are signed into a same account as the first computer system” includes computer systems that are signed into the same Wi-Fi network as the first computer system.
With that understanding in mind, Kim explicitly teaches that its list of pairable devices are signed into the same account, because the list of second devices 610 consists of second devices that responded (step 613) to the first device’s 600 device discover request message broadcast “via a network in step 607.” Kim ¶¶ 41–44. “Generally, examples of a protocol that provides device and service discover in an Internet Protocol (IP) network include a Universal Plug and Play (UPnP), Bonjour, Devices Profile for Web Services (DPWS), and such.” Kim ¶ 41.
Claim 116
Fitzpatrick and Kim teach the first computer system of claim 113, and Fitzpatrick further teaches:
displaying, via the display generation component, a quick response code that, when scanned by an external computer system, initiates a process for connecting the first computer system with the external computer system.
“The pairing routine may comprise an authorisation process to ensure that a user is authorised to use both the control device 5 and the capture device 10 in conjunction with one another,” which may include “a key or code exchange between the control device 5 and the capture device 10 in a way that maximises the likelihood that both devices are under the control of the same authorised user,” and more specifically, “a time-limited pseudo-randomly generated 2D machine-readable barcode may be displayed via the display unit 50 of the control device 5, that is to be captured and decoded by a camera of the capture device 10 for use in the pairing routine.” Fitzpatrick ¶¶ 153–155.
Fitzpatrick does not appear to explicitly disclose a “second connection user interface element” for triggering the display of its 2D machine-readable barcode.
Kim, however, teaches:
the list of one or more connection user interface elements includes a second connection user interface element, and the one or more programs further including instructions for: detecting, via the one or more input devices, a selection of the second connection user interface element;
The first device 600 detects “when selection of one device to be connected is input in the list of the devices according to a user input in step 619.” Kim ¶ 47.
and in response to detecting the selection of the second connection user interface element . . . initiate[] a process for connecting the first computer system with the external computer system.
As shown in FIG. 6, in response to selecting a device from the list, a process comprising steps 621–643 is performed, which involves pairing the devices. Kim ¶ 47.
This process does not necessarily involve a QR code exchange, but it was already established above that, under the combined system, one would follow Kim’s manual process for pairing a new device, and implement the actual pairing process in accordance with Fitzpatrick’s disclosure, which is an enhanced version of Kim’s pairing process that uses “a time-limited pseudo-randomly generated 2D machine-readable barcode.” See Fitzpatrick ¶ 155.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine Fitzpatrick and Kim for the reasons given in the rejections of claim 112 and 113 above. However, in addition, Fitzpatrick provides evidence that such a person of ordinary skill in the art, while making this combination, would have been motivated to retain Fitzpatrick’s time-limited pseudo-randomly generated 2D machine-readable barcode in the final combination, because Fitzpatrick explains that the barcode “serves as a mechanism to ensure that both devices are physically close to one another, and so implicitly under the control of the same authorised user.” Fitzpatrick ¶ 155.
Claim 117
Fitzpatrick and Kim teach the first computer system of claim 113, and Fitzpatrick further teaches its one or more programs further including instructions for:
in accordance with a determination that the second computer system is in a predetermined position, connecting one or more camera sensors of the second computer system with the first computer system.
“The pairing routine may comprise an authorisation process to ensure that a user is authorised to use both the control device 5 and the capture device 10 in conjunction with one another,” which may include “a key or code exchange between the control device 5 and the capture device 10 in a way that maximises the likelihood that both devices are under the control of the same authorised user,” and more specifically, “a time-limited pseudo-randomly generated 2D machine-readable barcode may be displayed via the display unit 50 of the control device 5, that is to be captured and decoded by a camera of the capture device 10 for use in the pairing routine.” Fitzpatrick ¶¶ 153–155.
Fitzpatrick does not appear to explicitly disclose selecting the first connection user interface element in order to initiate the determination of the camera capture process.
Kim, however, teaches:
detecting, via the one or more input devices, a selection of the first connection user interface element;
The first device 600 detects “when selection of one device to be connected is input in the list of the devices according to a user input in step 619.” Kim ¶ 47.
and after detecting selection of the first connection user interface element, [initiating the process for pairing the devices]
As shown in FIG. 6, in response to selecting a device from the list, a process comprising steps 621–643 is performed, which involves pairing the devices. Kim ¶ 47.
This process does not necessarily involve a QR code exchange, but it was already established above that, under the combined system, one would follow Kim’s manual process for pairing a new device, and implement the actual pairing process in accordance with Fitzpatrick’s disclosure, which is an enhanced version of Kim’s pairing process that uses “a time-limited pseudo-randomly generated 2D machine-readable barcode.” See Fitzpatrick ¶ 155.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine Fitzpatrick and Kim for the reasons given in the rejections of claim 112 and 113 above. However, in addition, Fitzpatrick provides evidence that such a person of ordinary skill in the art, while making this combination, would have been motivated to retain Fitzpatrick’s time-limited pseudo-randomly generated 2D machine-readable barcode in the final combination, because Fitzpatrick explains that the barcode “serves as a mechanism to ensure that both devices are physically close to one another, and so implicitly under the control of the same authorised user.” Fitzpatrick ¶ 155.
Claim 121
Fitzpatrick and Kim teach the first computer system of claim 112, the one or more programs further including instructions for:
receiving an indication that the second computer system is connected to the first computer system;
“Following connection, the video control program 61 and app 21 coordinate with one another to allow exchange of data via the local connection 6. This includes the exchange of command data, video stream data, and also technical capability data.” Fitzpatrick ¶ 157.
and in response to receiving the indication that the second computer system is connected to the first computer system, displaying, via the display generation component, a representation of video captured by one or more camera sensors of the second computer system.
“The capture device 10 can thus be effectively utilised as an efficient high-quality independent webcam,” Fitzpatrick ¶ 162, including “Facetime®.” Fitzpatrick ¶ 121.
Claim 122
Fitzpatrick and Kim teach the first computer system of claim 121, the one or more programs further including instructions for:
displaying, via the display generation component, a user interface of the application, including displaying, in the user interface of the application, one or more control user interface elements that, when selected, cause the first computer system to perform respective functions associated with the video captured by one or more camera sensors of the second computer system.
“[I]t is primarily envisaged that a user can operate the control device 5 to control the settings of the capture device 10. This conveniently removes the need for the user to interact with the capture device 10 via its own GUI, as presented by the relatively small screen 11.” Fitzpatrick ¶ 158. As one example, “the user interface of the app 21 and/or video control program 61 provides the user with a resolution adjustment element, such as a slider, allowing the user to specify minimum and/or maximum resolutions.” Fitzpatrick ¶ 171; see also Fitzpatrick Clm. 3.
Claim 129
Claim 129 recites a non-transitory computer readable storage medium that stores exactly the same programs as are stored on the memory of the system recited in claim 112. Since Fitzpatrick and Kim teach the memory in the system of claim 112 and all of its stored programs, and since a memory is but one example of a non-transitory computer readable storage medium (i.e., a memory is a species of the genus of computer readable media), claim 129 is rejected over all of the findings set forth in the rejection of claim 112. See MPEP § 2131.02 (subsection II.).
Claim 130
Claim 130 recites the exact same method that the system of claim 112 performs as part of its normal operation. “Under the principles of inherency, if a prior art device, in its normal and usual operation, would necessarily perform the method claimed, then the method claimed will be considered to be anticipated by the prior art device.” MPEP § 2112.02. Therefore, claim 130 is rejected over all of the findings and rationale as provided above for claim 112.
II. Fitzpatrick, Kim, and Devine teach claim 115.
Claim 115 is rejected under 35 U.S.C. § 103 as being unpatentable over Fitzpatrick and Kim, as applied to claim 113 above, and further in view of U.S. Patent Application Publication No. 2021/0255826 A1 (“Devine”).
Claim 115
Fitzpatrick and Kim teach the first computer system of claim 113, and Fitzpatrick further contemplates that the capture device 10 to be used with the control device 5 will be used in a mounted position. Fitzpatrick ¶ 163. That said, neither reference explicitly anticipates that the list of one or more connection user interface elements includes connection user interface elements that correspond to respective computer systems that are within a predetermined distance of the first computer system.
Devine, however, teaches a system that displays a list of one or more connection user interface elements representing devices in communication with one or more sensors, with which to connect to a first computer system,
wherein the list of one or more connection user interface elements includes connection user interface elements that correspond to respective computer systems that are within a predetermined distance of the first computer system.
“At FIG. 18E, in response to detecting (e.g., receiving the indication of) input 1829, and based on (e.g., in response to, in accordance with) the determination that there is are multiple recognized (e.g., previously-paired) devices 800a, 800b that are within the predefined proximity of computer system 1600, computer system 1600 causes device display 1601 to replace display of home screen user interface 1810 with disambiguation user interface 1830. Disambiguation user interface 1830 includes three selectable options 1832a, 1832b, 1832c. Option 1832a is associated with (e.g., corresponds to) user 1806b and device 800b, and is presented based on a determination that device 800b is within the predefined proximity of computer system 1600.” Devine ¶ 597.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to improve Fitzpatrick and Kim’s user interfaces for connecting to network devices with Devine’s technique of limiting the list of available devices to those within a predefined proximity of the pairing computer. One would have been motivated to utilize the techniques of Devine’s disambiguation user interface, because those techniques “reduce the cognitive burden on a user and produce a more efficient human-machine interface.” Devine ¶ 5.
III. Carrigan and Yang teach claim 124.
Claim 124 is rejected under 35 U.S.C. § 103 as being unpatentable over Carrigan as applied to claim 112 above, and further in view of U.S. Patent Application Publication No. 2018/0101297 A1 (“Yang”).
Claim 124
Carrigan teaches the first computer system of claim 112, the one or more programs further including instructions for:
“As illustrated in FIG. 8E, device 600 displays user interface 808 with notification 810.” Carrigan ¶ 310.
It is submitted that a person of ordinary skill in the art would understand the user interface 808 illustrated by FIG. 8E to be the lock screen for device 600, and that both lock screens and device notifications are operating system functions. Carrigan does not explicitly disclose the foregoing, and therefore, the second half of this rejection will provide evidence from the Yang reference as to why it was known and obvious to display a menu of notifications in a lock screen (and to explicitly request display of the same via a user input).
including displaying a camera user interface element;
FIG. 8E further illustrates that notification 810 “represents the front door camera (e.g., via representative image 810a from the front door camera and text ‘FRONT DOOR’).” Carrigan ¶ 310.
detecting, via the one or more input devices, a selection of the camera user interface element;
“At FIG. 8F, device 600 receives (e.g., detects) user input 850c (e.g., a tap) corresponding to selection of notification 810.” Carrigan ¶ 311.
and in response to detecting the selection of the camera user interface element, displaying, via the display generation component, a list of connection user interface elements for connecting the first computer system with respective computer systems, the list of connection user interface elements including the first connection user interface element.
“In response to receiving user input 850c, device 600 displays an image from the front door camera (e.g., a live or paused video stream, a playing or paused recorded video clip, or a notification of a recorded video clip).” Carrigan ¶ 311. The image (which is the same as the representation discussed in the rejection of claim 112) is provided amongst a list of user interface elements (affordances 814a–814c) that are selectable to perform operations over a connection with multiple other corresponding devices. See Carrigan ¶ 312.
As mentioned above, Carrigan does not explicitly say whether user interface 808 is “system-level,” though the illustration of user interface 808 in FIG. 8E as a lock screen strongly suggests the same. Carrigan also does not explicitly say how to reach the lock screen, let alone via “detecting, via the one or more input devices, a request to display a system-level control menu.”
Yang, however, teaches
detecting, via the one or more input devices, a request to display a system-level control menu;
“Push button 206 is, optionally, used . . . to lock the device by depressing the button and releasing the button before the predefined time interval has elapsed; and/or to unlock the device or initiate an unlock process.” Yang ¶ 148. When locked, the device may display a “locked state user interface,” e.g., as shown in FIG. 5A–5D. Yang ¶ 193. The locked state user interface, its menu of notifications, and the notifications themselves, are all provided by the device’s operating system. See, e.g., Yang ¶¶ 281 and 283.
in response to detecting the request to display the system-level control menu, displaying the system-level control menu,
“[I]n response to receiving the user input to activate the device from the dormant state, the device displays the updated first notification or concurrently displays the first and second notifications in accordance with whether the second communication and the first communication are received from the same sender.” Yang ¶ 256.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention, given everything disclosed in both of the above references, to display a lock screen with a list of notifications in response to a user input, thereby displaying notifications about the various cameras in the claimed “system-level control menu.” One would have been motivated to display the camera notifications in the system-level control menu of a lock screen because the lock screen provides increased security and privacy for a user, Yang ¶¶ 193 and 322, as well as energy savings. Yang ¶¶ 5–6 and 247.
IV. Su and Rauenbuehler teach claims 126 and 127.
Claims 126 and 127 are rejected under 35 U.S.C. § 103 as being unpatentable over Su as applied to claim 125 above, and further in view of U.S. Patent Application Publication No. 2019/0342449 A1 (“Rauenbuehler”).
Claim 126
Su teaches the first computer system of claim 125, but does not explicitly disclose whether “a fourth computer system displays an accept option for connecting the fourth computer system with the first computer system,” in addition to the second computer system.
Rauenbuehler, however, teaches a system that receives indications of at least a second computer system being selected for a communication from a first computer system,
wherein in response to receiving an indication that a user associated with the second computer system has been selected at the first computer system, a fourth computer system displays an accept option for connecting the fourth computer system with the first computer system.
“[A] call or other connection or alert/notification coming in to user device 310, such as using WWAN connection 308, can be relayed to a local server or other grouped device managing entity, such as using a WLAN connection 306 to a local access point 302. The incoming call or connection/alert/notification can then be broadcast from the local server or other grouped device managing entity to all of the user devices 310 and 320, such as using various WLAN connections 306, upon which all of the user devices may then ring, flash lights, and/or provide other alerts (visual, audible, and/or haptic) that the call or connection/alert/notification is incoming and can be answered at or responded to at any alert providing user device.” Rauenbuehler ¶ 39.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to improve Su’s calling interface with Rauenbuehler’s technique of notifying all of a user’s devices when one of the devices is receiving an incoming communication. One would have been motivated to improve Su’s system with Rauenbuehler’s technique because it helps prevent the inconvenience of missing the call from the first device. See Rauenbuehler ¶ 5.
Claim 127
Su and Rauenbuehler teach the first computer system of claim 126, wherein
in response to a determination that the accept option for connecting the second computer system with the first computer system has been selected, the fourth computer system ceases display of the accept option for connecting the fourth computer system with the first computer system.
“In the event that the incoming connection is answered (or responded to) at the user device 310 or any of the associated user devices 320, then the actual answering device can typically send a “connection answered” notification or status update so that all of the other user devices can stop ringing and/or providing other incoming connection alerts.” Rauenbuehler ¶ 40.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Justin R. Blaufeld whose telephone number is (571)272-4372. The examiner can normally be reached M-F 9:00am - 4:00pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James K Trujillo can be reached at (571) 272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Justin R. Blaufeld
Primary Examiner
Art Unit 2151
/Justin R. Blaufeld/Primary Examiner, Art Unit 2151
1 FIG. 12D contains a typographical error for this reference numeral as “12050c”
Prosecution Insights