DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice of Pre-AIA or AIA Status
This is the first Non-Final Office Action in response to Application Serial Number: 18/424,544, filed on January 26, 2024. Claims 1-20 are pending in this application and have been rejected below.
Priority
The Examiner has noted this Application claims priority from Provisional Application No. 63/529.484 filed July 28, 2023 and is a Continuation in Part (CIP) of Application No. 17/944,913 filed September 2022, which is a Continuation (CON) of Application No. 17/351,982 filed June 18, 2021, which claims priority from Provisional Application No. 63/208,739 filed on June 09, 2021 and Provisional Application No. 63/040,908 filed on June 18, 2020.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Step 1: The claimed subject matter falls within the four statutory categories of patentable subject matter.
Claims 1-10 are directed towards a method, claims 11-19 are directed towards a system and claim 20 is directed towards a non-transitory computer readable medium, which are among the statutory categories of invention.
Step 2A – Prong One: The claims recite an abstract idea.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite updating checklist items to resolve issues with elements of a worksite based on evaluation videos and pictures documented at the worksite.
Claim 1 recites limitations directed to an abstract idea based on mental processes and certain methods of organizing human activity. Specifically, presenting a checklist item; receiving an evaluation video documenting an element of a worksite corresponding to the checklist item; receiving an evaluation picture from the evaluation video, wherein the evaluation picture documents an element issue, the element issue portraying an issue with the element of the worksite documented by the evaluation video; receiving contextual data of the evaluation picture, wherein the contextual data of the evaluation picture comprises markup data, the markup data identifying the element issue documented by the evaluation picture; using the evaluation video and the evaluation picture to update the checklist item to an evaluated checklist item; and performing work at the worksite to resolve the issue with the element of the worksite based on the evaluated checklist item constitutes methods based on observations, evaluations, judgements and/or opinion that can be performed mentally by a combination of the human mind and a human using pen and paper, as well as, managing personal behavior. The claim does not recite any additional elements to take the claim out of the mental processes or certain methods of organizing human activity groupings. Thus the claim recites an abstract idea. The system comprising at least one processor and a non-transitory computer readable medium comprising an application executable by the at least on processor recited in Claim 11 and the non-transitory computer readable medium comprising instructions executable by processors recited in claim 20 do not take the claims out of the mental processes or certain methods of organizing human activity groupings. Thus, claims 11 and 20 recite certain mental processes and certain methods of organizing human activity for similar reasons as claim 1.
Step 2A – Prong Two: The judicial exception is not integrated into a practical application.
The judicial exception is not integrated into a practical application. Claim 1 does not recite any additional elements to analyze under Step 2A – Prong Two. The system comprising at least one processor and a non-transitory computer readable medium comprising an application executable by the at least on processor in Claim 11 and the non-transitory computer readable medium comprising instructions executable by processors in claim 20 are recited at a high-level of generality such that they amounts to no more than generic computer components used as tools to apply the instructions of the abstract idea; see MPEP 2106.05(f). Thus, the additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limitations on practicing the abstract idea. Claims 11 and 20 as a whole, looking at the additional elements individually and in combination, does not integrate the judicial exception into a practical application and therefore are directed to an abstract idea.
Step 2B: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional elements in the claims other than the abstract idea per se, including the system comprising at least one processor and a non-transitory computer readable medium comprising an application executable by the at least on processor and the non-transitory computer readable medium comprising instructions executable by processors amount to no more than a recitation of generic computer elements utilized to perform generic computer functions, such as receiving or transmitting data over a network, e.g., using the Internet to gather data, buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network); electronic recordkeeping, Ultramercial, 772 F.3d at 716, 112 USPQ2d at 1755 (updating an activity log) and storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93; see MPEP 2106.05(d)(II). Viewed as a whole, these additional claim elements do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Therefore, since there are no limitations in the claim that transform the abstract idea into a patent eligible application such that the claim amounts to significantly more than the abstract idea itself, the claims are rejected under 35 U.S.C. § 101 as being directed to non-statutory subject matter.
§ 101 Analysis of the dependent claims.
Regarding the dependent claims, dependent claims 3 and 13 recite watermark and overlay limitations that are not technological in nature and merely limits the abstract idea to a particular environment. Claims 6, 7, 9, 16, 17 and 19 recite presenting limitations, which are limitations considered to be an insignificant extra-solution activity of collecting and delivering data; see MPEP 2106.05(g). Claims 4 and 14 recite a user interface, claims 5 and 15 recite an evaluation device to record the video, claims 8 and 18 recite an interface element and claim 10 recites a thumbnail, all of which merely recite generic computer components used to apply the instructions of the abstract idea; MPEP 2106.05(f). Additionally, claims 2, 3, 6, 12, 13 and 16 recite steps that further narrow the abstract idea constituting mental processes and certain methods of organizing human activity. Therefore claims 2-10 and 12-19 do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 4-7, 11, 14-17 and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Winterstein et al., U.S. Publication No. 2021/0240987 [hereinafter Winterstein].
Referring to Claim 1, Winterstein teaches:
A method comprising:
presenting a checklist item (Winterstein, [0055]), “the user of client station 101 may begin the task of documenting on-site items identified by the user while at construction site 102 by launching a software tool on client station 101 that incorporates the disclosed front-end software component… such a software tool could take the form of a “Punch List” tool for documenting and managing punch items, an “Observations” tool for documenting and managing observation items, an “Incidents” tool for documenting and managing incident items, a “Daily Log” tool for documenting and managing daily-log items, or a “Task” tool for documenting and managing task items”; (Winterstein, [0078]; [0119]);
receiving an evaluation video documenting an element of a worksite corresponding to the checklist item (Winterstein, [0089]), “back-end platform 103 may perform an evaluation of a received piece of media content and then use this evaluation as a basis for breaking up the received piece of media content into discrete, item-specific segments of media content… if a received piece of media content comprises an audiovisual recording, back-end platform 103 may perform an evaluation of the video data included in such a recording in order to identify times when there was a threshold change in the scene depicted in the video (which are presumed to correspond to times when the user switched from capturing one on-site item to another) and then use these identified times as the division points for the received piece of media content”;
receiving an evaluation picture from the evaluation video, wherein the evaluation picture documents an element issue, the element issue portraying an issue with the element of the worksite documented by the evaluation video (Winterstein, [0113]), “if the captured media content that is descriptive of an on-site item comprises an audiovisual recording, back-end platform 103 may be configured to extract one or more images from the audiovisual recording that provide a visual representation of the on-site item. In this respect, back-end platform 103 may be configured to either (i) capture an image at a predefined point within an audiovisual recording for an on-site item (e.g., the halfway marker of the audiovisual recording) or (ii) dynamically determine the point within an audiovisual recording for an on-site item at which to capture an image based on an analysis of the video data (e.g., an analysis of which point provides a “best view” of the on-site item)”; (Winterstein, [0091]), “back-end platform 103 may translate each item-specific piece of media content for an on-site item… back-end platform 103 may optionally be configured to apply image recognition technology to the video data (e.g., object classification and/or detection models) and then generate text that describes the results of that image recognition (e.g., textual descriptions of object classes and/or specific objects that are recognized in the video data)”; (Winterstein, [0064]-[0065]), “UI view 500 includes (i) a selectable “Start Recording” button 501 that may be selected by the user in order to instruct client station 101 to start capturing a first audiovisual recording for a first on-site item identified by the user, which is shown to be an issue with the caulking of a bath tub located in the master bathroom of construction site 102…”; (Winterstein, [0138]; [0092]);
receiving contextual data of the evaluation picture, wherein the contextual data of the evaluation picture comprises markup data, the markup data identifying the element issue documented by the evaluation picture (Winterstein, [0139]), “…an “Image” field that has been automatically populated with an extracted screenshot of the punch item, along with the audiovisual recording of the second punch item that was originally captured by client station 101 …”; (Winterstein, [0137]), “UI views that may be presented to a user for purposes of reviewing, editing, and approving generated data records are illustrated in FIG. 7. In particular, as shown, FIG. 7 illustrates (i) an example of a first UI view 700 that shows a list of unapproved data records that have been generated by back-end platform 103 for punch items that were identified by the user at construction site 102, as well as (ii) an example of a second UI view 701 that shows the details of the unapproved data record for the second punch item shown in the list of UI view 700, which may be presented to the user when that second punch item is selected by the user in UI view 700”; (Winterstein, [0074]), “client station 101 may be configured to receive user input indicating a location of an on-site item before, during, and/or after capturing the media content that is descriptive of the on-site item, in which case client station 101 may use this user-based location information to supplement the captured media content that is descriptive of the identified on-site item. In this example implementation, the user input indicating a location of an on-site item may take any of various forms, examples of which may include a textual identification of the location that is typed by the user, a selection of a location within a 2D drawing of construction site 102 that is presented to the user, and/or a verbal identification of the location that is spoken by the user (which could correspond to a particular location in a 2D drawing or the like), among other possibilities”; (Winterstein, [0098]; [0065]);
using the evaluation video and the evaluation picture to update the checklist item to an evaluated checklist item (Winterstein, [0141]-[0142]), “UI view 800 that shows both (i) a list of unapproved data records that have been generated by back-end platform 103 for punch items that were identified by the user at construction site 102 and (ii) the details of the unapproved data record for whichever punch item is currently selected within the presented list of unapproved data records…”; (Winterstein, [0138]; [0148]); and
performing work at the worksite to resolve the issue with the element of the worksite based on the evaluated checklist item (Winterstein, [0149]), “Once the data records generated by back-end platform 103 for the on-site items have been approved and made available to the full universe of users that generally have access to data records of this type, such users may then interact with those data records to manage and resolve the on-site items as appropriate”.
Referring to Claim 4, Winterstein teaches the method of claim 1. Winterstein further teaches:
further comprising: receiving a selection of the checklist item presented in a user interface (Winterstein, [0136]), “client station 101 may initially present the user with an option for viewing a list of unapproved data records that have been generated by back-end platform 103 for the on-site items identified by the user while at construction site 102, and when this option is selected, client station 101 may present a first UI view that shows the list of unapproved data records. From this list, the user can input a selection of a given one of the unapproved data records for an on-site item identified by the user at construction site 102 (e.g., via a touch screen or mouse), which may then trigger client station 101 to present a second UI view showing the details of the selected data record—including the information that has been extracted and automatically populated into the data fields of the selected data record by back-end platform 103…”; (Winterstein, [0137]; [0139]; [0143]).
Referring to Claim 5, Winterstein teaches the method of claim 1. Winterstein further teaches:
further comprising: presenting a camera view responsive to a selection of the checklist item, wherein the camera view is displayed on an evaluation device to record the evaluation video (Winterstein, [0064]-[0065]), “(i) a selectable “Start Recording” button 501 that may be selected by the user in order to instruct client station 101 to start capturing a first audiovisual recording for a first on-site item identified by the user, which is shown to be an issue with the caulking of a bath tub located in the master bathroom of construction site 102, and (ii) an unhighlighted timer 502 with a value of 00:00:00, which indicates that client station 101 is not currently engaging in an active media capture session… the user may then select the “Start Recording” button 501, and in response to detecting this user input, client station 101 may function to (i) update UI view 500 to indicate that client station 101 has begun capturing the first audiovisual recording for the first on-site item… a highlighted timer 504 with a value of 00:00:04, which indicates that client station 101 is actively capturing the first audiovisual recording and has currently recorded 4 seconds of audiovisual content that is descriptive of the first on-site item”; (Winterstein, [0057]), “client station 101 may be used to capture different types of media content that is descriptive of the identified on-site items. As one example, the user may use client station 101 to capture one or more audio recordings while the user verbally describes the on-site items identified at construction site 102. As another example, the user may use client station 101 to capture one or more audiovisual recordings while the user verbally describes the on-site items identified at construction site 102 and also points the client station's camera at certain aspects of construction site 102 that are associated with the identified on-site items. Client station 101 could be used to capture other types of media content is descriptive of the identified on-site items as well” .
Referring to Claim 6, Winterstein teaches the method of claim 1. Winterstein further teaches:
further comprising: presenting a playback view responsive to a selection of the checklist item, wherein the playback view displays (Winterstein, Fig. 7, item 701; [0139]), “UI view 701… and an “Image” field that has been automatically populated with an extracted screenshot of the punch item, along with the audiovisual recording of the second punch item that was originally captured by client station 101”.
Referring to Claim 7, Winterstein teaches the method of claim 1. Winterstein further teaches:
further comprising:
presenting a camera view to record the evaluation video responsive to a selection of the checklist item; and presenting embedded information in (Winterstein, [0064]-[0065]), “(i) a selectable “Start Recording” button 501 that may be selected by the user in order to instruct client station 101 to start capturing a first audiovisual recording for a first on-site item identified by the user, which is shown to be an issue with the caulking of a bath tub located in the master bathroom of construction site 102, and (ii) an unhighlighted timer 502 with a value of 00:00:00, which indicates that client station 101 is not currently engaging in an active media capture session… the user may then select the “Start Recording” button 501, and in response to detecting this user input, client station 101 may function to (i) update UI view 500 to indicate that client station 101 has begun capturing the first audiovisual recording for the first on-site item… a highlighted timer 504 with a value of 00:00:04, which indicates that client station 101 is actively capturing the first audiovisual recording and has currently recorded 4 seconds of audiovisual content that is descriptive of the first on-site item”; (Winterstein, [0057]), “client station 101 may be used to capture different types of media content that is descriptive of the identified on-site items. As one example, the user may use client station 101 to capture one or more audio recordings while the user verbally describes the on-site items identified at construction site 102. As another example, the user may use client station 101 to capture one or more audiovisual recordings while the user verbally describes the on-site items identified at construction site 102 and also points the client station's camera at certain aspects of construction site 102 that are associated with the identified on-site items. Client station 101 could be used to capture other types of media content is descriptive of the identified on-site items as well”.
Referring to Claim 11, Winterstein teaches:
A system comprising at least one processor (Winterstein, [0044]; [0050]); and
a non-transitory computer readable medium comprising an application that, when executing on the at least one processor (Winterstein, [0045]; [0051]), performs:
Claim 11 disclose substantially the same subject matter as claim 1, and is rejected using the same rationale as previously set forth.
Claim 14 disclose substantially the same subject matter as claim 4, and is rejected using the same rationale as previously set forth.
Claim 15 disclose substantially the same subject matter as claim 5, and is rejected using the same rationale as previously set forth.
Claim 16 disclose substantially the same subject matter as claim 6, and is rejected using the same rationale as previously set forth.
Claim 17 disclose substantially the same subject matter as claim 7, and is rejected using the same rationale as previously set forth.
Referring to Claim 20, Winterstein teaches:
A non-transitory computer readable medium comprising instructions that, when executed by one or more processors (Winterstein, [0045]; [0051]), perform:
Claim 20 disclose substantially the same subject matter as claim 1, and is rejected using the same rationale as previously set forth.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 8-10, 12, 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Winterstein et al., U.S. Publication No. 2021/0240987 [hereinafter Winterstein], and further in view of Jobe, U.S. Publication No. 2014/0288984 [hereinafter Jobe].
Referring to Claim 2, Winterstein teaches the method of claim 1. Winterstein teaches a user of client station is able to review data records that are generated based on the captured media content (see par. 0128) and users may then interact with those data records to manage and resolve the on-site items (see par. 0149), but Winterstein does not explicitly teach:
further comprising:
receiving a review video;
receiving a review picture from the review video, wherein the review picture documents a resolution of the element issue documented by the evaluation picture;
receiving contextual data of the review picture, wherein the contextual data of the review picture comprises markup data of the review picture, the markup data identifying the resolution of the element issue, wherein the markup data of the review picture contrasts with the markup data of the evaluation picture; and using the review video and the review picture to update the evaluated checklist item to a reviewed checklist item.
However Jobe teaches:
further comprising:
receiving a review video (Jobe, [0046]), “if an administrator reviews the results and determines that the cleanliness rating for a given store location is low and/or he/she views photos or video from the store location and determines that the store appears less clean than is desired, he/she may send directions to the store to address the situation”; (Jobe, [0042]; [0044]);
receiving a review picture from the review video, wherein the review picture documents a resolution of the element issue documented by the evaluation picture; and receiving contextual data of the review picture, wherein the contextual data of the review picture comprises markup data of the review picture, the markup data identifying the resolution of the element issue, wherein the markup data of the review picture contrasts with the markup data of the evaluation picture; and using the review video and the review picture to update the evaluated checklist item to a reviewed checklist item (Jobe, [0025]), “Users may respond to assigned tasks by providing input from the field, including but not limited to, providing photos, video, audio… users may respond to assigned tasks providing input from the field, including but not limited to, providing photos, video, audio, and/or answers to questions. In block 105, administrators or other specified users may have the ability to view a dashboard of results as input is provided from the field in real-time so that incorrect results may be promptly corrected as described in more detail later. In additional or other embodiments of the present disclosure, administration system 10 may provide for users to be reassigned to different routes or geographical areas. Results may be restricted or filtered with respect to a particular area or product type as described in more detail below. Also, as will be described in more detail below, system 10 may provide the ability to restrict if a user can create tasks or just review tasks within system 10”; (Jobe, [0022]), “These tasks may include but are not limited to due dates, instructions, videos, audio recordings and/or documents to provide guidance for proper completion of a task”; (Jobe, [0046]-[0047]), “, if an administrator reviews the results and determines that the cleanliness rating for a given store location is low and/or he/she views photos or video from the store location and determines that the store appears less clean than is desired, he/she may send directions to the store to address the situation… incorrect or undesirable results may be corrected according to an embodiment of the present disclosure. In step 601, if something is wrong in the entry of information related to an assigned task, an administrator may provide correction associated with an incorrect answer or visual information collected in the field. In step 602, tools may be provided within the administration system so that an administrator may draw on photos that have been transmitted through the administration system to identify corrections to be made in some embodiments of the present disclosure. In additional or other embodiments of the present disclosure, an administrator may transmit a message to a user over a wireless communication network that may provide textual corrections to be made and/or indications as to what might need to be fixed in the field …” (Jobe, [0040]; [0024]; [0030]; [0044]; [0048]).
At the time the invention was filed, it would have been obvious to a person of ordinary skill in the art to have modified the review of data records and resolving of on-site items in Winterstein to include the review video and picture limitations as taught by Jobe. The motivation for doing this would have been to improve the method of analyzing media content captured at a construction site in Winterstein (see par. 0013) to efficiently include the results of verifying and remediating assigned tasks (see Jobe par. 0002).
Referring to Claim 8, Winterstein teaches the method of claim 1. Winterstein further teaches:
further comprising: presenting a verification view using the evaluation video (Winterstein, [0143]), “in UI view 800, the presented details of the unapproved data record for the first punch item…UI view 800 also includes a selectable “Confirm” button 802, which the user can select to approve the unapproved data record for the first punch item after the user has finished reviewing and potentially editing that data record”; (Winterstein, [0139]), “details of the unapproved data record… and an “Image” field that has been automatically populated with an extracted screenshot of the punch item, along with the audiovisual recording…”.
Winterstein teaches a selectable confirm button to approve the data record for a punch list (see par. 0143), where the data record includes an image field populated with a screenshot and audiovisual recording (see par. 0139), but Winterstein does not explicitly teach:
the evaluation picture with an interface element to initiate recording a review video.
However Jobe teaches
the evaluation picture with an interface element to initiate recording a review video (Jobe, [0047]), “an administrator may provide textual and/or photographic instructions to the user to direct him/her how to correct the store display or the product placement. Audio and/or video instructions also may be provided in some embodiments of the present disclosure”; (Jobe, [0040]), “if a user assigned to one or more tasks nears the location where the assigned task(s) may be performed, he/she may select the assignment to be completed on his/her display screen. In step 403, the user may respond to all assigned tasks, including but not limited to, being given the ability to fill in the blank to answer a question, scan/read a bar code, answer other questions, take photos, and record video and/or audio”; (Jobe, [0048]).
At the time the invention was filed, it would have been obvious to a person of ordinary skill in the art to have modified the approved punch-list item in Winterstein to include the picture and video limitations as taught by Jobe. The motivation for doing this would have been to improve the method of analyzing media content captured at a construction site in Winterstein (see par. 0013) to efficiently include the results of verifying and remediating assigned tasks (see Jobe par. 0002).
Referring to Claim 9, Winterstein teaches the method of claim 1. Winterstein teaches a selectable confirm button to approve the data record for a punch list (see par. 0143), where the data record includes an image field populated with a screenshot and audiovisual recording (see par. 0139), but Winterstein does not explicitly teach:
further comprising: presenting a verification view using the evaluation video, the evaluation picture, a review video, and a review picture.
However Jobe teaches:
further comprising: presenting a verification view using the evaluation video, the evaluation picture, a review video, and a review picture (Jobe, [0025]), “Users may respond to assigned tasks by providing input from the field, including but not limited to, providing photos, video, audio, and answers to questions, in block 103. It should be appreciated that answers to questions may be provided in a variety of different forms according to embodiments of the present disclosure. For example, answers may be provided in a yes/no format, multiple choice, and/or as a textual input without departing from the present disclosure. Block 103 may be referred to as responding to a task. In block 104, users may respond to assigned tasks providing input from the field, including but not limited to, providing photos, video, audio, and/or answers to questions… administrators or other specified users may have the ability to view a dashboard of results as input is provided from the field in real-time so that incorrect results may be promptly corrected as described in more detail later”; (Jobe, [0022]), “verifying and remediating an assigned task… these tasks may include but are not limited to due dates, instructions, videos, audio recordings and/or documents to provide guidance for proper completion of a task… prompting the user to complete a specified task by a specific date/time and/or at a specified location. Further embodiments of the present disclosure may provide systems and methods to direct corrective action to be taken with respect to one or more specified tasks”; (Jobe, [0045]).
At the time the invention was filed, it would have been obvious to a person of ordinary skill in the art to have modified the approved punch-list item in Winterstein to include the picture and video limitations as taught by Jobe. The motivation for doing this would have been to improve the method of analyzing media content captured at a construction site in Winterstein (see par. 0013) to efficiently include the results of verifying and remediating assigned tasks (see Jobe par. 0002).
Referring to Claim 10, Winterstein teaches the method of claim 1. Winterstein teaches a selectable confirm button to approve the data record for a punch list (see par. 0143), where the data record includes an image field populated with a screenshot and audiovisual recording (see par. 0139), but Winterstein does not explicitly teach:
further comprising:
presenting a verification view using the evaluation picture and a review picture with embedded data overlaid onto a thumbnail of the evaluation picture and a thumbnail of the review picture.
However Jobe teaches:
further comprising:
presenting a verification view using the evaluation picture and a review picture with embedded data overlaid onto a thumbnail of the evaluation picture and a thumbnail of the review picture (Jobe, [0025]), “users may respond to assigned tasks providing input from the field, including but not limited to, providing photos, video, audio, and/or answers to questions. In block 105, administrators or other specified users may have the ability to view a dashboard of results as input is provided from the field in real-time so that incorrect results may be promptly corrected”; (Jobe, [0033]), “an administrator may be presented with a plurality of icons from which to select… a photo/video/audio input (represented by the camera icon in FIG. 3b).
At the time the invention was filed, it would have been obvious to a person of ordinary skill in the art to have modified the approved punch-list item in Winterstein to include the picture and video limitations as taught by Jobe. The motivation for doing this would have been to improve the method of analyzing media content captured at a construction site in Winterstein (see par. 0013) to efficiently include the results of verifying and remediating assigned tasks (see Jobe par. 0002).
Claim 12 disclose substantially the same subject matter as claim 2, and is rejected using the same rationale as previously set forth.
Claim 18 disclose substantially the same subject matter as claim 8, and is rejected using the same rationale as previously set forth.
Claim 19 disclose substantially the same subject matter as claim 9, and is rejected using the same rationale as previously set forth.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Winterstein et al., U.S. Publication No. 2021/0240987 [hereinafter Winterstein], in view of Lorenzo, U.S. Publication No. 2018/0300552 [hereinafter Lorenzo], and further in view of Mosier, U.S. Publication No. 2015/0186894 [hereinafter Mosier].
Referring to Claim 3, Winterstein teaches the method of claim 1. Winterstein teaches extracting one or more images from the audiovisual recording that provide a visual representation of the on-site item (see par. 0113), but Winterstein does not explicitly teach:
further comprising:
receiving the evaluation picture, wherein the evaluation picture is adjusted to include the markup data of the evaluation picture, one or more watermarks of the evaluation picture, and one or more overlays of the evaluation picture, the overlays of the evaluation picture comprising overlay data of the evaluation picture; and
receiving a review picture, wherein the review picture is adjusted to include markup data of the review picture, one or more watermarks of the review picture, and one or more overlays of the review picture, the overlays of the review picture comprising overlay data of the review picture.
However Lorenzo teaches:
further comprising:
receiving the evaluation picture, wherein the evaluation picture is adjusted to include the markup data of the evaluation picture, , and one or more overlays of the evaluation picture, the overlays of the evaluation picture comprising overlay data of the evaluation picture (Lorenzo, [0030]), “allows remote review of 360 degree building photographs in order to monitor building construction progress, identify problems during construction”; (Lorenzo, [0081]), “Second 360 degree photos 516 are taken at a later time than first 360 degree photos 508, and may be expected to reflect construction progress since a time when the first 360 degree photos 508 were obtained”; (Lorenzo, [0049]), “360 degree photo metadata 304 may include a 360 degree photo time 316, identified as 360 degree photo time 316a through 360 degree photo time 316n. The 360 degree photo time 316 represents the time when a first or second 360 degree photo was taken”; (Lorenzo, [0030]), “annotate photographs to either describe the problem or propose a solution”; (Lorenzo, [0051]), “Annotation is any form, combination, and quantity of text, symbols, or audio associated with a 360 degree photo. Annotation specifies one or more of an action, a construction state, a construction error, a date, a time, or a reminder, and each such annotation may be different in content and form from other annotations”; (Lorenzo, [0076]-[0078]), “photo annotations”; (Lorenzo, [0034]-[0036]); and
receiving a review picture, wherein the review picture is adjusted to include markup data of the review picture, and one or more overlays of the review picture, the overlays of the review picture comprising overlay data of the review picture (Lorenzo, [0076]), “Annotation 712 may also include descriptive graphics such as a directional arrow or a circled item within the annotated first 360 degree photo 512”; (Lorenzo, [0079]), “the application 216 may highlight one or more parts of a photo 508, 512, 516 or add annotation 712 noting a delay in the project”; (Lorenzo, [0092]), “The highlights may be actual highlights applied to one or more of first 360 degree photos 508, 512 and the second 360 degree photos 516, one or more overlays applied to the first 360 degree photos 508, 512 and the second 360 degree photos 516, new or different colors applied to the first 360 degree photos 508, 512 and the second 360 degree photos 516, or any other technique for annunciation of the difference”; (Lorenzo, [0030]), “annotate photographs to either describe the problem or propose a solution”; (Lorenzo, [0051]), “Annotation is any form, combination, and quantity of text, symbols, or audio associated with a 360 degree photo. Annotation specifies one or more of an action, a construction state, a construction error, a date, a time, or a reminder, and each such annotation may be different in content and form from other annotations”; (Lorenzo, [0076]-[0078]), “photo annotations”.
At the time the invention was filed, it would have been obvious to a person of ordinary skill in the art to have modified the images extracted from videos in Winterstein to include the picture limitations as taught by Lorenzo. The motivation for doing this would have been to improve the method of analyzing media content captured at a construction site in Winterstein (see par. 0013) to efficiently include the results of tracking progress at a building construction site using photos (see Lorenzo par. 0009).
Winterstein teaches extracting one or more images from the audiovisual recording that provide a visual representation of the on-site item (see par. 0113) and Lorenzo teaches building photographs for monitoring building construction progress (see par. 0030), However Winterstein in view of Lorenzo does not explicitly teach:
one or more watermarks of the evaluation picture and one or more watermarks of the review picture.
However Mosier teaches:
one or more watermarks of the evaluation picture and one or more watermarks of the review picture (Mosier, [0071]-[0072]), “… documentation and other evidence showing that the buried utility, or utilities, have been physically marked. The documentation may include taking digital photographs of the markings and/or the surrounding environment… the location services client may annotate the photographs, such as by watermarking or in any other fashion, so that it will be apparent as to which ticket the photographs are associated with… the photographs can be annotated with other information as well including, but not limited to, GPS coordinates of the specific location from which the photograph was taken, the name of the locator, and the date and time the photograph was taken…”.
At the time the invention was filed, it would have been obvious to a person of ordinary skill in the art to have modified images extracted from videos in Winterstein and photographs in Lorenzo to include the watermark limitations as taught by Mosier. The motivation for doing this would have been to improve the method of analyzing media content captured at a construction site in Winterstein (see par. 0013) and tracking progress at a building construction site using photos in Lorenzo (see par. 0009) to efficiently include the results of enabling photographs associated with a particular ticket to be retrieved as necessary (see Mosier par. 0071).
Claim 13 disclose substantially the same subject matter as claim 3, and is rejected using the same rationale as previously set forth.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Johnson (US 20150332203 A1) – System and method for remote selection and management of projects by residential property managers and third party vendor service providers. The system provides for a digital means of combining the project selection, project management, and payment aspects of the residential service industry into a single method.
Burke et al. (US 20130124242 A1) – Subject matter disclosed herein relates to video content editing, and in particular, to video review workflow.
Nixon et al. (US 10152031 B2) – A method executed in a process control environment automatically generates, displays, and tracks check lists according to assigned work items. A check list is displayed on a user interface device, which tracks, internally or in cooperation with a supervisor engine, whether a user of the user interface device has checked off the items on the check list.
Edwards (WO 2005033844 A2) - Job site communications systems, methods, and computer program products. According to one embodiment of the subject matter disclosed herein, a job site communications system is provided. The job site communications system can include a wireless communications device adapted to be positioned at a job site and operable to wirelessly communicate annotated computer-aided design documents from the job site. The job site communications system can also include a computer positioned at a site remote from a job site and operable to receive or transmit the computer-aided design documents.
Lin et al. (Understanding On-Site Inspection of Construction Projects Based on Keyword Extraction and Topic Modeling) – A novel text mining approach based on keyword extraction and topic modeling is introduced to identify key concerns and their dynamics of on-site issues for better decision-making process. Then, the proposed approach was demonstrated in a real world project and tested with 7250 issue records. Results showed that the proposed method could successfully extract key concerns hidden in texts and identify their changes with time, thereby enabling a more efficient on-site inspection and data-centric decision-making process. This research contributes: (1) to the body of knowledge a new framework for discovering key concerns and their changes with time in texts, and (2) to the state of practice by providing insights on hot topics and their changes with time to reduce on-site issues and make decisions efficiently.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Crystol Stewart whose telephone number is (571)272-1691. The examiner can normally be reached 9:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Patty Munson can be reached at (571)270-5396. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CRYSTOL STEWART/Primary Examiner, Art Unit 3624