DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/1/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered and attached by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
Image capturing device configured to in claims 1, 2, 3, and 9 described in ¶16.
Electronic labeling device configured to in claims 1, 2, 3, and 5 described in ¶16.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-11 and 13-20 are rejected under 35 U.S.C. 103 as being unpatentable over Jaime (U.S. Patent Pub. No. 2015/0262617) in view of Judi (U.S. Patent Pub. No. 2023/0063386).
Regarding Claim 1, Jaime teaches a system, comprising:
an image capturing device configured to capture a set of images based on a first set of user inputs, wherein an image of the set of images comprises image timing information (Jaime 2015/0262617 ¶39 When one or more activity indicator is selected by a user, the media clip creation application notes which activity indicator(s) has/have been selected, and, typically, the time that the selection occurred, sometimes referred to herein as an "activity indicator selection time". This information may be used to create a media clip of the activity at a later time. For example, the media clip creation application may create a media clip immediately after an activity indicator is selected while the user is still recording. In another embodiment, one or more media clips are created after the user has finished recording, using activity indicator selection times stored by content capture and tagging device 100;)
an electronic labeling device configured to generate a set of labels based on a second set of user inputs, wherein a label of the set of labels comprises label timing information (Fig. 8, ¶160 content tagging devices 802a and 802b generate tagging information (activity identification information) as predefined activities occur during the event; ¶52 As media clips of the soccer game are created by the media clip creation application, they may be associated with activity identification information, such as one or more of an event code, an event type, an activity type associated with each activity indicator, a time and date that the activity or event occurred or a time that an activity indicator was selected by a user, one or more team names, one or more names of persons appearing in the activity, the name of the user recording the event, etc;)
a memory for storing machine-readable instructions; and a processor core for accessing the machine-readable instructions and executing the machine-readable instructions as operations, the operations comprising (¶55 Server 104 comprises a digital computing device executing machine-readable instructions stored in a memory that, when executed, cause server 104 to receive media clips from content capture and tagging devices 100:)
accessing the set of images and the set of labels (¶39 one or more media clips are created after the user has finished recording, using activity indicator selection times stored by content capture and tagging device 100;)
determining a time relationship between the image capturing device and the electronic labeling device with a time of day (¶39 The activity indicator selection time is generally the time of day, normally available to content capture and tagging device 100, or provided by server 104, or by one or more other wireless networks. In case content capture and tagging device 100 is not within range of a wireless network and therefore not able to receive accurate timing information, content capture device may assign an elapsed time from the start of the primary media recording to each activity tagged by selection of an activity indicator, then replace the elapsed times with "time of day" times when content capture and tagging device 100 re-enters coverage;)
associating the label with the image based on the image timing information, the label timing information, and the time relationship; and outputting an indication of the association of the label with the image (¶39 when content capture and tagging device 100 is within range of a wireless network, it may synchronize to a server clock, then maintain this clock if content capture and tagging device 100 moves out of range with the wireless network; ¶40 In another embodiment, content capture and tagging device 100 may not have an ability to receive time of day information, for example, in an application where content capture and tagging device 100 comprises a video camera. In this case, activity indicator selection times may be determined by filming a "clapboard" displaying the current time of day and a code correlated to the event as the video camera begins recording. Then, activity indicator selection times may be assigned to tagged activities based on an elapsed time from when filming began.)
Jaime does not explicitly disclose determining a time relationship between an image capturing device and an electronic labeling device and associating a label with an image based on image timing information, label timing information, and the time relationship.
Judi is in the same field of art of image analysis. Further, Judi teaches determining a time relationship between an image capturing device and an electronic labeling device and associating a label with an image based on image timing information, label timing information, and the time relationship (¶84 Time stamps are added to the recording video by the respective controllers of the eyewear 100 and UAV 800. In accordance with this example, the moment the user of eyewear 100 stops recording, the UAV 800 stops recording too. The controller 932 resolves any latency issues while creating the time stamps, e.g., through the synchronization or by adjusting for detected latency.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Jaime by determining a relationship between a camera and the labeling device that is taught by Judi; thus, one of ordinary skilled in the art would be motivated to combine the references the control a UAV (Judi ¶85).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 2, Jaime in view of Judi discloses the system of claim 1, wherein the electronic labeling device generates a synchronization output, the image capturing device captures a synchronization data point based on the synchronization output, and the time relationship is determined based on the synchronization output and the synchronization data point (Jaime, ¶45 Each activity indicator may have its own time duration or "time window" associated with it, pre-assigned in accordance with an expected or typical time duration of each associated activity. For example, while the home run activity indicator may be predefined to create media clips each having a time duration of 17 seconds with a "pre-activity time" of 7 seconds and a "post-activity time" of 10 seconds, a "double play" activity indicator 208 may be pre-assigned a "pre-event time" of 6 seconds and only 1 second of "post-activity time".) (Judi 2023/0063386 ¶84 Time stamps are added to the recording video by the respective controllers of the eyewear 100 and UAV 800. In accordance with this example, the moment the user of eyewear 100 stops recording, the UAV 800 stops recording too. The controller 932 resolves any latency issues while creating the time stamps, e.g., through the synchronization or by adjusting for detected latency.)
The reasons for combining Jaime and Judi are similar to that stated in the rejection of claim 1. In addition, this same reasoning is pertinent and applicable to the rejections of claims 3, 6, 9, and 11 below.
Regarding Claim 3, Jaime in view of Judi discloses the system of claim 2, wherein the time relationship between the image capturing device and the electronic labeling device is a time offset between first timing information associated with the synchronization output and second timing information associated with the synchronization data point (Judi, ¶84 Time stamps are added to the recording video by the respective controllers of the eyewear 100 and UAV 800. In accordance with this example, the moment the user of eyewear 100 stops recording, the UAV 800 stops recording too. The controller 932 resolves any latency issues while creating the time stamps, e.g., through the synchronization or by adjusting for detected latency; ¶26 The processor compensates for any latency between when the eyewear begins recording and when the UAV begins recording.)
Regarding Claim 4, Jaime in view of Judi discloses the system of claim 1, wherein associating the label with the image is further based on a time threshold for label selection (Jaime, ¶45 Each activity indicator may have its own time duration or "time window" associated with it, pre-assigned in accordance with an expected or typical time duration of each associated activity. For example, while the home run activity indicator may be predefined to create media clips each having a time duration of 17 seconds with a "pre-activity time" of 7 seconds and a "post-activity time" of 10 seconds, a "double play" activity indicator 208 may be pre-assigned a "pre-event time" of 6 seconds and only 1 second of "post-activity time".)
Regarding Claim 5, Jaime in view of Judi discloses the system of claim 1, wherein the label is selected from a set of displayed labels on the electronic labeling device (Jaime, ¶36 activity indicators 204, 206, 208, 210, 212, and 218 are overlaid in a semi-transparent manner over an electronic representation of the baseball game as it is recorded and displayed on user interface 404. In other embodiments, the activity indicators may be hidden from view until the user touches a predetermined portion of user interface 404; ¶38 the user may select all three of these activity indicators after the grand slam has been hit.)
Regarding Claim 6, Jaime in view of Judi discloses the system of claim 1, wherein the image is a first image of the set of images and the image timing information is first image timing information, and a second image of the set of images is associated with the label based on second image timing information, the label timing information, and the time relationship (Judi ¶26 teaches compensating between two different video streams/images taken form eyewear and a UAV. This is simply a design choice a person having ordinary skill in the art could arrange the timing information being based off either image/video and having the label associated with the other image/video.)
Regarding Claim 7, Jaime in view of Judi discloses the system of claim 1, wherein associating the label with the image is further based on an estimated user delay associated with a user (Jaime, ¶26 The processor compensates for any latency between when the eyewear begins recording and when the UAV begins recording; ¶69 FIG. 8B, one example of the eyewear 100 controlling the UAV 800 is shown. The eyewear 100 is coupled to the UAV 800 via a wireless connection 802 to send control signals from the eyewear 100 to the UAV 800 as a function of eyewear user input.)
Regarding Claim 8, Jaime in view of Judi discloses the system of claim 1, wherein the operations further comprise receiving user feedback to the association of the label with the image (Jaime, ¶22 in response to user selection of an activity indicator, a media clip is created, showing the activity that occurred. The media clip may be indexed, associated, or annotated, with tagging information such as metadata, tags, keys, and/or other information so that it may be easily discoverable by others online; ¶97 in response to the activity occurring, the user selects one of the activity indicators corresponding to the activity. For example, if a soccer goal was scored, the user may select a "Goal" activity indicator displayed on user interface 404.)
Regarding Claim 9, Jaime in view of Judi discloses the system of claim 1, wherein the image capturing device is on an unmanned aerial vehicle (UAV) controlled via the first set of user inputs (Judi, ¶105 At block 1006, when the user initiates the recording in block 1004, the processor 932 of eyewear 100 simultaneously instructs the UAV 800 to begin recording the objects 884 viewed by camera 820. The processor 932 instructs the UAV 800 via wireless connection 802. The processor 932 may delay the recording using cameras 114 a short time period corresponding to the known latency for the UAV 800 to begin recording.)
Regarding claim 10, claim 10 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Jaime further teaching on: A non-transitory machine-readable medium having machine executable instructions for an image classification system that causes a processor core to execute operations (¶216 an embodiment of the invention may comprise a non-transitory machine-readable media embodying code or machine-readable instructions to implement the teachings, methods, processes, algorithms, steps and/or functions disclosed herein.)
Regarding Claim 11, Jaime in view of Judi discloses the non-transitory machine-readable medium of claim 10, wherein the operations further comprise accessing a synchronization image of a time-dependent pattern, wherein the synchronization image comprises synchronization timing information, and the time relationship is determined based on a first time determined from the time-dependent pattern and a second time determined from the synchronization timing information (Judi, ¶84 The controller 932 resolves any latency issues while creating the time stamps, e.g., through the synchronization or by adjusting for detected latency; ¶85 Utilizing the eyewear 100 and the UAV 800 forms a “movie crew in the box,” as the processor 932 produces a stitched video 890 with 1st and 3rd person perspective that is synchronized second by second. The stitched video 890 can be displayed on the displays 180C and 180D, where display 180D is shown in FIG. 8J, or on a display remote from the eyewear 100. The eyewear video frames are displayed on the left side of the display 180D, and the UAV video frames are displayed on the right side of the display 180D. The frames are synchronized in time using the respective timestamps.)
Claim 13 recites limitations similar to claim 4 and is rejected under the same rationale and reasoning.
Claim 14 recites limitations similar to claim 8 and is rejected under the same rationale and reasoning.
Regarding Claim 15, Jaime in view of Judi discloses the non-transitory machine-readable medium of claim 14, wherein the label is a first label and wherein the operations further comprise associating a second label with the image based on the user feedback (Jaime, ¶22 in response to user selection of an activity indicator, a media clip is created, showing the activity that occurred. The media clip may be indexed, associated, or annotated, with tagging information such as metadata, tags, keys, and/or other information so that it may be easily discoverable by others online; ¶97 in response to the activity occurring, the user selects one of the activity indicators corresponding to the activity. For example, if a soccer goal was scored, the user may select a "Goal" activity indicator displayed on user interface 404.)
Regarding claim 16, claim 16 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Jaime further teaching on: A method (¶20 The present application describes various embodiments of media clip creation and distribution systems, apparatus and methods)
Claims 17 recites limitations similar to claim 3 and is rejected under the same rationale and reasoning.
Claims 18 recites limitations similar to claim 3 and is rejected under the same rationale and reasoning.
Claim 19 recites limitations similar to claim 4 and is rejected under the same rationale and reasoning.
Regarding Claim 20, Jaime in view of Judi discloses the method of claim 16, wherein the image is a first image of a set of images, the image timing information is first image timing information, and the time relationship is a first time relationship, the method further comprising: accessing a second image from the set of images, the second image comprising second image timing information from the memory; determining a second time relationship between the second image and the label based on the second image timing information and the label timing information; associating the label with the second image based on the second time relationship to generate a second labeled image comprising the second image and the label; and outputting the second labeled image (This is just a reiteration of the independent claim, Jaime teaches in ¶103 Of course, multiple media clips may be created during a single recording, e.g., the time between when the user begins recording until the time that the user stops recording. As indicated above, the media clips may be created "on-the-fly" or after the user stops recording.)
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Jaime (U.S. Patent Pub. No. 2015/0262617) in view of Judi (U.S. Patent Pub. No. 2023/0063386) in view of (U.S. Patent Pub. No. 2019/0124254).
Regarding Claim 12, Jaime in view of Judi teaches the non-transitory machine-readable medium of claim 11.
Jaime in view of Judi does not explicitly disclose wherein the time-dependent pattern comprises a quick-response (QR) code.
Shibahara is in the same field of art of image analysis. Further, Shibahara teaches wherein the time-dependent pattern comprises a quick-response (QR) code (¶119 Server synchronization processor 122c performs processing of synchronizing photographing system clock 111c of terminal device 110C with clock 121 of server 120 by such a technique as the NTP; ¶120 Time code coder 125 codes current clock time TSSVR based on clock 121 of server 120 into the time code in each predetermined period. For example, the time code is a two-dimensional code, such as a QR code (registered trademark), which indicates the clock time. The time code may be any image constructed with a pattern, a character, a numerical character, or a symbol as long as the time code visually indicates the clock time.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Jaime in view of Judi by using a QR code for a time pattern that is taught by Shibahara; thus, one of ordinary skilled in the art would be motivated to combine the references for easily matching the photographing clock time with the desired timing with high accuracy (Shibahara ¶8).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUSTIN BILODEAU whose telephone number is (571)272-1032. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUSTIN BILODEAU/Examiner, Art Unit 2664
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights