Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The Action is responsive to the Amendments and Remarks filed on 9/25/2025. Claims 1, 3-4, 6-11, and 14-16 are pending claims. Claims 1, 15, and 16 are written in independent form. Claims 2, 5, and 12-13 have been cancelled.
Priority
Acknowledgment is made of a claim for priority as a continuation of PCT/JP2021/031985, filed 08/31/2021, which claims foreign priority to JP2020-167558, filed 10/2/2020, under 35 U.S.C. § 119(a)-(d) or (f), and is also acknowledged. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
Claims 1 and 14 are objected to because of the following informalities:
Claim 1 appears to recite a typographical error by first reciting “acquire…inspection record data created in a predetermined format in which text data of predetermined information is input…” and then reciting “analyze the text data of the inspection report to specify a member…, a position…, and a type…corresponding to the text data” and it is unclear which text data is being referred to by “…corresponding to the text data”. The language is being understood as intended to recite “…corresponding to the text data of the inspection report”.
Claim 14 appears to recite a typographical error by reciting “acquire…inspection record data created in a predetermined format in which text data of predetermined information is input for each of a plurality of inspection points of the structure” in Independent Claim 1 and “Wherein the inspection record data is created in the predetermined format in which the text data of information of the member of the structure, the damage to the structure, the position of the damage, and findings of the damage is input for each of the plurality of inspection points.” in Dependent Claim 14. The language of Dependent Claim 14 is being understood as intended to recite “Wherein the inspection record data is created in the predetermined format in which the text data of predetermined information of the member of the structure, the damage to the structure, the position of the damage, and findings of the damage is input for each of the plurality of inspection points.”
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3-4, 6-11, and 14-16 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-patentable subject matter. The claimed invention is directed to one or more abstract ideas without significantly more. The judicial exception is not integrated into a practical application. The claims do not include additional elements that are sufficient to amount to significantly more than judicial exception. The eligibility analysis in support of these findings is provided below.
As per Independent Claim 1,
STEP 1:In accordance with Step 1 of the eligibility inquiry (as explained in MPEP 2106), the claimed device (claims 1, 3-4, 6-11, and 14), device (claim 15), and device (claim 16) are directed to one of the eligible categories of subject matter and therefore satisfies Step 1.
STEP 2A Prong One:The independent claim 1 recites the following limitations directed to an abstract idea:
Receive selection of an inspection record from the inspection record data;
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, observing a selection of an inspection record from the inspection record data, thus receiving the selection through observation.
Analyze the text data of the inspection record to specify a member of the structure, a position of damage to the structure, and a type of the damage to the structure corresponding to the text data;
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by observing and evaluating the text data of the inspection record, and based on the observation and evaluation, making a judgement and/or opinion of a specific member of the structure, a position of damage to the structure, and a type of the damage to the structure corresponding to the text data.
Extract a portion of the three-dimensional model data corresponding to the member, the position of the damage, and the type of the damage;
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by observing and evaluating the member, the position of the damage, the type of the damage, and the three-dimensional model data, and based on the observation and evaluation, making a judgement and/or opinion of a portion of the three-dimensional model data that corresponds to the member, the position of the damage, and the type of the damage.
Extract a portion of the inspection data associated with the portion of the three-dimensional model data;
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by observing and evaluating the portion of the three-dimensional model data and the inspection data, and based on the observation and evaluation, making a judgement and/or opinion of a portion of the inspection data associated with the portion of the three-dimensional model data.
STEP 2A Prong Two:Claims 1 recites that the steps are performed using “an inspection support device”, “a processor”, and “a display device”, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
The claim recites the following additional elements:
Acquire three-dimensional model data of the structure, inspection data of the structure associated with the three-dimensional model data, and inspection record data,
The limitation recites an insignificant extra solution activity as retrieval of data (ie. Mere data gathering) such as ‘obtaining information’ as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
The inspection record data created in a predetermined format in which text data of predetermined information is input for each of a plurality of inspection points of the structure;
The limitation recites an insignificant extra-solution activity as selecting a particular type of data being used to represent the inspection record data as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
Display the inspection record data on a display device;
The limitation recites using “a display device” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
Display the portion of the three-dimensional model data and the portion of the inspection data on the display device.
The limitation recites using “a display device” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
Viewing the additional limitations together and the claim as a whole, nothing provides integration into a practical application.
STEP 2B:
The conclusions for the mere implementation using a computer are carried over and does not provide significantly more.
With respect to “Acquire three-dimensional model data of the structure, inspection data of the structure associated with the three-dimensional model data, and inspection record data,” identified as insignificant extra-solution activity above this is also WURC as court-identified see MPEP 2106.05(d)(II)(i).
With respect to “The inspection record data created in a predetermined format in which text data of predetermined information is input for each of a plurality of inspection points of the structure;” identified as insignificant extra-solution activity above this is also WURC as court-identified see MPEP 2106.05(d)(II)(iv).
Looking at the claim as a whole does not change this conclusion and the claim is ineligible.
As per Dependent Claims 3-4, 6-11, and 14,
STEP 1:In accordance with Step 1 of the eligibility inquiry (as explained in MPEP 2106), the claimed device (claims 1, 3-4, 6-11, and 14), device (claim 15), and device (claim 16) are directed to one of the eligible categories of subject matter and therefore satisfies Step 1.
STEP 2A Prong One:The dependent claims 3-4, 6-11, and 14 recite the following limitations directed to an abstract idea:
The limitation of Dependent Claim 4 includes the step(s) of:
Wherein the processor is configured to
Map the portion of the inspection data to the portion of the three-dimensional model data,
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by observing and evaluating the portion of the inspection data and the portion of the three-dimensional model data, and based on the observation and evaluation, making a judgement and/or opinion to map/associate the portion of the inspection data with the portion of the three-dimensional model data.
The limitation of Dependent Claim 11 includes the step(s) of:
Wherein the processor is configured to:
Analyze the text data of the inspection record to extract past inspection data corresponding to the member, the position of the damage, and the type of the damage; and
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by observing and evaluating the text data of the inspection record, the member, the position of the damage, and the type of the damage, and based on the observation and evaluation, making a judgement and/or opinion of past inspection data corresponding to the member, the position of the damage, and the type of the damage.
The limitation of Dependent Claim 14 includes the step(s) of:
Wherein the inspection record data is created in the predetermined format in which the text data of information of the member of the structure, the damage to the structure, the position of the damage, and findings of the damage is input for each of the plurality of inspection points.
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by observing and evaluating a predefined format and text data of information of the member of the structure, the damage to the structure, the position of the damage, and findings of the damage for each of the plurality of inspection points, and based on the observation and evaluation, using pen and paper to create an inspection record in the predetermined format with the text data of information of the member of the structure, the damage to the structure, the position of the damage, and findings of the damage is input for each of the plurality of inspection points.
STEP 2A Prong Two:The claim(s) recite the following additional elements:
The limitation of Dependent Claim 3 includes the step(s) of:
A memory configured to store the three-dimensional model data, the inspection data associated with the three-dimensional model data, and the inspection record data,
The limitation recites using “a memory” to store data, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
Wherein the processor is configured to acquire the three-dimensional model data, the inspection data, and the inspection record data from the memory.
The limitation recites an insignificant extra solution activity as retrieval of data (ie. Mere data gathering) such as ‘obtaining information’ as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
The limitation of Dependent Claim 4 includes the step(s) of:
Display the portion of the inspection data mapped to the portion of the three-dimensional model data on the display device.
The limitation recites using “the display device” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
The limitation of Dependent Claim 6 includes the step(s) of:
Wherein the three-dimensional model data includes at least a member region of the structure and data of a member of the structure.
The limitation recites an insignificant extra-solution activity as selecting a particular type of data being used to represent the three-dimensional model data as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
The limitation of Dependent Claim 7 includes the step(s) of:
Wherein the inspection data includes a plurality of types of data.
The limitation recites an insignificant extra-solution activity as selecting a particular type(s) of data being used to represent the inspection data as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
The limitation of Dependent Claim 8 includes the step(s) of:
Wherein the plurality of types of data include a captured image, a panoramic composite image, damage information, and a two-dimensional drawing.
The limitation recites an insignificant extra-solution activity as selecting a particular types of data being used to represent the inspection data as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
The limitation of Dependent Claim 9 includes the step(s) of:
Wherein the processor is configured to display at least one type of data on the display device from the plurality of types of data included in the inspection.
The limitation recites using “the display device” and “the processor” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
The limitation of Dependent Claim 10 includes the step(s) of:
Wherein the inspection data includes a plurality of captured images; and
The limitation recites an insignificant extra-solution activity as selecting a particular types of data being used to represent the inspection data as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
Wherein the processor is configured to display the captured image satisfying a condition on the display device from the plurality of captured images to be displayed.
The limitation recites using “the display device” and “the processor” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
The limitation further recites an insignificant extra-solution activity as selecting a particular type of data being used to represent the captured image being displayed (as a captured image that has satisfied a condition) as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
The limitation of Dependent Claim 11 includes the step(s) of:
The limitation of Dependent Claim 11 includes the step(s) of:
Wherein the processor is configured to:
display the past inspection data on the display device.
The limitation recites using “the display device” and “the processor” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
Viewing the additional limitations together and the claim as a whole, nothing provides integration into a practical application.
STEP 2B:
The conclusions for the mere implementation using a computer are carried over and does not provide significantly more.
With respect to Claim 3 reciting “Wherein the processor is configured to acquire the three-dimensional model data, the inspection data, and the inspection record data from the memory.” identified as insignificant extra-solution activity above this is also WURC as court-identified see MPEP 2106.05(d)(II)(i).
With respect to Claim 6 reciting “Wherein the three-dimensional model data includes at least a member region of the structure and data of a member of the structure.” identified as insignificant extra-solution activity above this is also WURC when claimed in a merely generic manner as court-identified see MPEP 2106.05(d)(II)(iv).
With respect to Claim 7 reciting “Wherein the inspection data includes a plurality of types of data.” identified as insignificant extra-solution activity above this is also WURC when claimed in a merely generic manner as court-identified see MPEP 2106.05(d)(II)(iv).
With respect to Claim 8 reciting “Wherein the plurality of types of data include a captured image, a panoramic composite image, damage information, and a two-dimensional drawing.” identified as insignificant extra-solution activity above this is also WURC when claimed in a merely generic manner as court-identified see MPEP 2106.05(d)(II)(iv).
With respect to Claim 10 reciting “Wherein the inspection data includes a plurality of captured images;” identified as insignificant extra-solution activity above this is also WURC when claimed in a merely generic manner as court-identified see MPEP 2106.05(d)(II)(iv).
With respect to Claim 10 reciting “the captured image satisfying a condition” identified as insignificant extra-solution activity above this is also WURC when claimed in a merely generic manner as court-identified see MPEP 2106.05(d)(II)(iv).
Looking at the claim as a whole does not change this conclusion and the claim is ineligible.
As per Independent Claim 15,
STEP 1:In accordance with Step 1 of the eligibility inquiry (as explained in MPEP 2106), the claimed device (claims 1, 3-4, 6-11, and 14), device (claim 15), and device (claim 16) are directed to one of the eligible categories of subject matter and therefore satisfies Step 1.
STEP 2A Prong One:The independent claim 15 recites the following limitations directed to an abstract idea:
receive selection of a text data of at least one of the plurality of inspection points from the list of text data;
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, observing a selection of text data of at least one of the plurality of inspection points from the list of text data, thus receiving the selection through observation.
analyze the text data to extract the inspection data corresponding to the text data of the at least one of the plurality of inspection points;
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by observing and evaluating the text data of the inspection record, and based on the observation and evaluation, making a judgement and/or opinion of a specific inspection data corresponding to the text data of the at least one of the plurality of inspection points.
STEP 2A Prong Two:Claims 15 recites that the steps are performed using “an inspection support device”, “a processor”, and “a display device”, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
The independent claim 15 recites the following additional elements:
Acquire three-dimensional model data of the structure, inspection data associated with the three-dimensional model data, and a list of text data of a plurality of inspection points related to an inspection work of the structure,
The limitation recites an insignificant extra solution activity as retrieval of data (ie. Mere data gathering) such as ‘obtaining information’ as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
Wherein the inspection data include:
a captured image obtained by imaging the structure, or
the captured image and a damage detection result image in which a damage detected from the captured image is expressed in a method of expressing according to a type of the damage;
The limitation recites an insignificant extra-solution activity as selecting a particular type of data being used to represent the inspection data as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
display the list of text data on a display device;
The limitation recites using “a display device” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
display on the display device, as inspection data, the three-dimensional model data to which the captured image is mapped or the captured image and the damage detection result image are mapped.
The limitation recites using “a display device” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
Viewing the additional limitations together and the claim as a whole, nothing provides integration into a practical application.
STEP 2B:
The conclusions for the mere implementation using a computer are carried over and does not provide significantly more.
With respect to “Acquire three-dimensional model data of the structure, inspection data associated with the three-dimensional model data, and a list of text data of a plurality of inspection points related to an inspection work of the structure,” identified as insignificant extra-solution activity above this is also WURC as court-identified see MPEP 2106.05(d)(II)(i).
With respect to “Wherein the inspection data include a captured image obtained by imaging the structure, or the captured image and a damage detection result image in which a damage detected from the captured image is expressed in a method of expressing according to a type of the damage;” identified as insignificant extra-solution activity above this is also WURC as court-identified see MPEP 2106.05(d)(II)(iv).
Looking at the claim as a whole does not change this conclusion and the claim is ineligible.
As per Independent Claim 16,
STEP 1:In accordance with Step 1 of the eligibility inquiry (as explained in MPEP 2106), the claimed device (claims 1, 3-4, 6-11, and 14), device (claim 15), and device (claim 16) are directed to one of the eligible categories of subject matter and therefore satisfies Step 1.
STEP 2A Prong One:The independent claim 16 recites the following limitations directed to an abstract idea:
receive selection of a text data of at least one of the plurality of inspection points from the list of text data;
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, observing a selection of text data of at least one of the plurality of inspection points from the list of text data, thus receiving the selection through observation.
analyze the text data to extract the inspection data corresponding to the text data of the at least one of the plurality of inspection points; and
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by observing and evaluating the text data of the inspection record, and based on the observation and evaluation, making a judgement and/or opinion of a specific inspection data corresponding to the text data of the at least one of the plurality of inspection points.
analyze the text data to determine whether the text data includes text data related to progressiveness of damage; and
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by observing and evaluating the text data, and based on the observation and evaluation, making a judgement and/or opinion as to whether the text data includes text data related to progressiveness of damage;
in response to determining that the text data include the text data related to the progressiveness of the damage,
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by making a judgement and/or opinion that the text data includes text data related to progressiveness of damage, and based on the judgement and/or opinion that the text data includes text data related to the progressiveness of damage, performing another mental process of observation, evaluation, judgement, and/or opinion.
extract past inspection data corresponding to the at least one of the plurality of inspection points,
The limitation recites a mental process of observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper, by observing and evaluating the at least one of the plurality of inspection points, and based on the observation and evaluation, making a judgement and/or opinion of past inspection data corresponding to the at least one of the plurality of inspection points.
STEP 2A Prong Two:Claims 16 recites that the steps are performed using “an inspection support device”, “a processor”, and “a display device”, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
The independent claim 16 recites the following additional elements:
Acquire three-dimensional model data of the structure, inspection data associated with the three-dimensional model data, and a list of text data of a plurality of inspection points related to an inspection work of the structure,
The limitation recites an insignificant extra solution activity as retrieval of data (ie. Mere data gathering) such as ‘obtaining information’ as identified in MPEP 2106.05(g) and does not provide integration into a practical application.
display the list of text data on a display device;
The limitation recites using “a display device” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
display the inspection data on the display device;
The limitation recites using “a display device” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
display the past inspection data on the display device.
The limitation recites using “a display device” to display information, which is a high-level recitation of generic computer components and represents mere instructions to apply on a computer as in MPEP 2106.05(f), which does not provide integration into a practical application.
Viewing the additional limitations together and the claim as a whole, nothing provides integration into a practical application.
STEP 2B:
The conclusions for the mere implementation using a computer are carried over and does not provide significantly more.
With respect to “Acquire three-dimensional model data of the structure, inspection data associated with the three-dimensional model data, and a list of text data of a plurality of inspection points related to an inspection work of the structure,” identified as insignificant extra-solution activity above this is also WURC as court-identified see MPEP 2106.05(d)(II)(i).
Looking at the claim as a whole does not change this conclusion and the claim is ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3-4, 6-11, and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Omansky et al. (U.S. Pre-Grant Publication No. 2014/0337286, hereinafter referred to as Omansky) and further in view of Liu (U.S. Pre-Grant Publication No. 2018/0025659).
Regarding Claim 15:
Omansky teaches an inspection support device for a structure, the inspection support device comprising a processor (Para. [0029]) configured to:
Acquire three-dimensional model data of the structure, inspection data associated with the three-dimensional model data, and a list of text data of a plurality of inspection points related to an inspection work of the structure,
Omansky teaches “Data structures and data sets of Building Information Model objects in the BIM Database Software Application 90 are transferred 92 to the Field Database Software Application 94. The unique data structures and data sets, related to one or may field operations, authored out in field, on the job site and at the point of construction, by construction field personnel, are then transferred 93 from the Field Database Software Application 94 back to the respective Building Information Model objects in the BIM Database Software Application 90.” (Para. [0049]) where “In the graphical user interface of the BIM Database Software Application 90, the data structures and data sets of Building Information Model objects are generally represented by visual or pictorial, three-dimensional (3D) shapes including building geometry and spatial relationships 91.” (Para. [0050]).Omansky further teaches inspection data by teaching “the data structures and data sets relate to building assemblies, materials, systems, sub-systems, equipment, components, and inter-relations thereof in a building project. In other embodiments, the data structures and data sets relate to at least one field process selected from the group consisting of systems commissioning, issue tracking, field reports, materials tracking, safety, quality assurance, quality control, work lists, punch lists, and handover.” (Para. [0010])Therefore Omansky teaches acquiring data structures represented by visual or pictoral, 3d shapes, inspection data sets such as field reports, quality assurance, etc., and text data of the data sets related to the inspection work of the building structure.
Wherein the inspection data include:
a captured image obtained by imaging the structure, or
Omansky teaches “Exemplary documents include operations and maintenance manuals, shop drawings, test reports, construction drawings, "as-builts" or as-installed drawings,” (Para. [0009]) thereby teaching capturing a plurality of documents as captured images in the form of drawings.
the captured image and a damage detection result image in which a damage detected from the captured image is expressed;
Omansky teaches “Exemplary documents include…emergency operating procedures, warranty, warranty guarantor, and spare parts lists” (Para. [0009]) which are all understood as information related to damage.
Display the list of text data on a display device;
Omansky teaches displaying the list of text data on a display device (Figure 5).
Display, on the display device, as the inspection data, the three-dimensional model data to which:
the captured image is mapped or
Omansky teaches Figure 10A as a screenshot, and thus is displaying, imported or received data structures and data sets (inspection data) which are “mapped to the objects in the field database and are linked to the database” (Paras. [0022] & [0034]). Omansky further teaches “In the graphical user interface of the BIM Database Software Application 90, the data structures and data sets of Building Information Model objects are generally represented by visual or pictorial, three-dimensional (3D) shapes including building geometry and spatial relationships 91.” (Para.[0050]) and Figure 10A also shows a three-dimensional model to which data is mapped being displayed.It is noted that Omansky teaches “Exemplary documents include operations and maintenance manuals, shop drawings, test reports, construction drawings, "as-builts" or as-installed drawings,” (Para. [0009]) thereby teaching capturing a plurality of documents as captured images that can be mapped and displayed as mapped inspection data.
the captured image and the damage detection result image are mapped.
Omansky teaches Figure 10A as a screenshot, and thus is displaying, imported or received data structures and data sets (inspection data) which are “mapped to the objects in the field database and are linked to the database” (Paras. [0022] & [0034]). Omansky further teaches “In the graphical user interface of the BIM Database Software Application 90, the data structures and data sets of Building Information Model objects are generally represented by visual or pictorial, three-dimensional (3D) shapes including building geometry and spatial relationships 91.” (Para.[0050]) and Figure 10A also shows a three-dimensional model to which data is mapped being displayed.It is noted that Omansky teaches ““Exemplary documents include operations and maintenance manuals, shop drawings, test reports, construction drawings, "as-builts" or as-installed drawings,…emergency operating procedures, warranty, warranty guarantor, and spare parts lists” (Para. [0009]) which are all understood as information related to damage, and therefore teaches capturing a plurality of documents as captured images and damage detection result images that can be mapped and displayed as mapped inspection data.
Omansky explicitly teaches all of the elements of the claimed invention as recited above except:
Receive selection of a text data of at least one of the plurality of inspection points from the list of text data;
Analyze the text data to extract the inspection data corresponding to the text data of the at least one of the plurality of inspection points;
However, in the related field of endeavor of linking relevant text information with virtual models, Omansky teaches:
Receive selection of a text data of at least one of the plurality of inspection points from the list of text data;
Liu teaches “a user input selecting a linked text displayed in the textual instruction section, identifying a component of the object in the 3D virtual model that is linked to the linked text, and highlighting, in response to identifying the component of the object in the 3D virtual model, the component of the object in the 3D virtual model displayed on display interface of the display device.” (Abstract)
Analyze the text data to extract the inspection data corresponding to the text data of the at least one of the plurality of inspection points;
Liu teaches “identifying a component of the object in the 3D virtual model that is linked to the linked text, and highlighting, in response to identifying the component of the object in the 3D virtual model, the component of the object in the 3D virtual model displayed on display interface of the display device.” (Abstract).
Thus, it would have been obvious to one of ordinary skill in the art, having the teachings of Liu and Omansky at the time that the claimed invention was effectively filed, to have combined the highlighting feature of a particular component when selecting a particular piece of text, as taught by Liu, with the systems and methods for construction field management and operations with building information modeling, as taught by Omansky.
One would have been motivated to make such combination because Omansky teaches mapping data structures to data sets and corresponding text in the data sets (Paras. [0012], [0034], & Fig. Fig. 5) and Liu teaches “The method may also include rotating the object in the virtual model section 104 to provide an improved view of the component in response to a selection of a linked text in the textual instruction section 102 of the electronic manual 100. The method may also include providing a zoomed in view of the component in response to a selection of a linked text in the textual instruction section 102 of the electronic manual 100. The method may also include tilting the object to provide an improved view of the component in response to a selection of a linked text in the textual instruction section 102 of the electronic manual 100. The method may also include highlighting a linked text in the textual instruction section 102 in response to a selection of a component of the object 106 in the virtual model section 104.” (Para. [0054]) and it would have been obvious to a person having ordinary skill in the art that providing improved views of particular objects in the displayed visual would improve the user’s ability to review and inspect the details about the object of interest.
Regarding Claim 1:
Omansky teaches an inspection support device for a structure comprising a processor (Para. [0029]) configured to:
Acquire three-dimensional model data of the structure, inspection data of the structure associated with the three-dimensional model data, and inspection record data created in a predetermined format in which text data of predetermined information is input for each of a plurality of inspection points of the structure,
Omansky teaches “Data structures and data sets of Building Information Model objects in the BIM Database Software Application 90 are transferred 92 to the Field Database Software Application 94. The unique data structures and data sets, related to one or may field operations, authored out in field, on the job site and at the point of construction, by construction field personnel, are then transferred 93 from the Field Database Software Application 94 back to the respective Building Information Model objects in the BIM Database Software Application 90.” (Para. [0049]) where “In the graphical user interface of the BIM Database Software Application 90, the data structures and data sets of Building Information Model objects are generally represented by visual or pictorial, three-dimensional (3D) shapes including building geometry and spatial relationships 91.” (Para. [0050]) and “The tabular format with text values in columns and rows 95 facilitates use by construction field personnel and generally lowers barriers to adoption. The tabular format is faster and easier to navigate, search and read through a large amount of information in a list format, filtered, grouped and sorted as required, than in a three-dimensional (3D) shapes format.” (Para. [0052]).
Omansky further teaches inspection data by teaching “the data structures and data sets relate to building assemblies, materials, systems, sub-systems, equipment, components, and inter-relations thereof in a building project. In other embodiments, the data structures and data sets relate to at least one field process selected from the group consisting of systems commissioning, issue tracking, field reports, materials tracking, safety, quality assurance, quality control, work lists, punch lists, and handover.” (Para. [0010])Therefore Omansky teaches acquiring data structures represented by visual or pictoral, 3d shapes, inspection data of the structure associated with the visual or pictoral 3d shapes, and inspection record data created in a predetermined format in which text data of predetermine dinformaiotn ins input for the inspection points, such as field reports, quality assurance, etc., and text data of the data sets related to the inspection work of the building structure.
Display the inspection record data on a display device;
Omansky teaches displaying the list of text data on a display device (Figure 5) where the text data is inspection record data by teaching “the data structures and data sets relate to building assemblies, materials, systems, sub-systems, equipment, components, and inter-relations thereof in a building project. In other embodiments, the data structures and data sets relate to at least one field process selected from the group consisting of systems commissioning, issue tracking, field reports, materials tracking, safety, quality assurance, quality control, work lists, punch lists, and handover.” (Para. [0010])
Analyze the text data of the inspection record to specify a member of the structure, a position of damage to the structure, and a type of the damage to the structure corresponding to the text data;
Omansky teaches 3d model data where “FIG. 9, which shows a representation of a user interface 120 for an example BIM database application with visualization of model objects from business rules and logic by the field database software application of the present invention” (Para. [0055]) and “the Status value 121 of each unique virtual model object in the Field database” (Para. [0055]) thereby teaching specifying the unique member of the structure, position of the structure, and corresponding text data in the Field database.
Omansky further teaches the structure data as being associated with damage to the structure by teaching “Exemplary documents include…emergency operating procedures, warranty, warranty guarantor, and spare parts lists” (Para. [0009]) which are all understood as information related to damage.
Display the portion of the three-dimensional model data and the portion of the inspection data on the display device.
Omansky teaches “transmitting the data structures and data sets includes transmitting data at a level of detail required for the field operations or processes” (Claim 5) where “FIG. 5 is a drawing showing a representation of graphical user interfaces and data displays of an example BIM database application and of the field database software application of the present invention” (Para. [0017]) and Figs. 5, 6, and 9 depict displaying corresponding portions of the 3d model data and/or a screen of the extracted/relevant inspection data “at a level of detail required for the field operations or processes”.
Omansky explicitly teaches all of the elements of the claimed invention as recited above except:
Receive selection of an inspection record from the inspection record data;
Extract a portion of the three-dimensional model data corresponding to the member, the position of the damage, and the type of the damage;
Extract a portion of the inspection data associated with the portion of the three-dimensional model data;
However, in the related field of endeavor of linking relevant text information with virtual models, Omansky teaches the processor:
Receive selection of an inspection record from the inspection record data;
Liu teaches “a user input selecting a linked text displayed in the textual instruction section, identifying a component of the object in the 3D virtual model that is linked to the linked text, and highlighting, in response to identifying the component of the object in the 3D virtual model, the component of the object in the 3D virtual model displayed on display interface of the display device.” (Abstract).
extract a portion of the three-dimensional model data corresponding to the member, the position of the damage, and the type of the damage;
Liu teaches “identifying a component of the object in the 3D virtual model that is linked to the linked text, and highlighting, in response to identifying the component of the object in the 3D virtual model, the component of the object in the 3D virtual model displayed on display interface of the display device.” (Abstract). Therefore, Liu teaches extracting both the portion of linked text and the corresponding portion of the object in the 3D virtual model.It is noted that the extracting is not performed using the corresponding information, just that the extracted portion corresponds to the member, the position of the damage, and the type of the damage.
Extract a portion of the inspection data associated with the portion of the three-dimensional model data;
Liu teaches “identifying a component of the object in the 3D virtual model that is linked to the linked text, and highlighting, in response to identifying the component of the object in the 3D virtual model, the component of the object in the 3D virtual model displayed on display interface of the display device.” (Abstract). Therefore, Liu teaches extracting both the portion of linked text and the corresponding portion of the object in the 3D virtual model.
Thus, it would have been obvious to one of ordinary skill in the art, having the teachings of Liu and Omansky at the time that the claimed invention was effectively filed, to have combined the highlighting feature of a particular component when selecting a particular piece of text, as taught by Liu, with the systems and methods for construction field management and operations with building information modeling, as taught by Omansky.
One would have been motivated to make such combination because Omansky teaches mapping data structures to data sets and corresponding text in the data sets (Paras. [0012], [0034], & Fig. Fig. 5) and Liu teaches “The method may also include rotating the object in the virtual model section 104 to provide an improved view of the component in response to a selection of a linked text in the textual instruction section 102 of the electronic manual 100. The method may also include providing a zoomed in view of the component in response to a selection of a linked text in the textual instruction section 102 of the electronic manual 100. The method may also include tilting the object to provide an improved view of the component in response to a selection of a linked text in the textual instruction section 102 of the electronic manual 100. The method may also include highlighting a linked text in the textual instruction section 102 in response to a selection of a component of the object 106 in the virtual model section 104.” (Para. [0054]) and it would have been obvious to a person having ordinary skill in the art that providing improved views of particular objects in the displayed visual would improve the user’s ability to review and inspect the details about the object of interest.
Regarding Claim 3:
Liu and Omansky further teach:
A memory configured to store the three-dimensional model data, the inspection data associated with the three-dimensional model data, and the inspection record data,
Omansky teaches “The unique data structures and data sets, related to one or may field operations, authored out in field, on the job site and at the point of construction, by construction field personnel, are then transferred from the Field Database 43 to the BIM Database 40 by the Integration Adapter 42 over the Internet 41, via either a wireless or wired connection. The unique data structures and data sets, related to one or may field operations, are associated with the unique Building Information Model objects representing building assemblies, systems, equipment and components.” (Para.[0044]).Omansky further explicitly teaches memory as part of teaching computer storage (Para. [0031]).
Wherein the processor acquires the three-dimensional model data, the inspection data, and the list of text data from the memory.
Omansky teaches “Data structures and data sets of Building Information Model objects in the BIM Database Software Application 90 are transferred 92 to the Field Database Software Application 94. The unique data structures and data sets, related to one or may field operations, authored out in field, on the job site and at the point of construction, by construction field personnel, are then transferred 93 from the Field Database Software Application 94 back to the respective Building Information Model objects in the BIM Database Software Application 90.” (Para. [0049]) where “In the graphical user interface of the BIM Database Software Application 90, the data structures and data sets of Building Information Model objects are generally represented by visual or pictorial, three-dimensional (3D) shapes including building geometry and spatial relationships 91.” (Para. [0050]).
Regarding Claim 4:
Liu and Omansky further teach wherein the processor is configured to:
Map the portion of the inspection data to the portion of the three-dimensional model data and
Omansky teaches “attributes from the Building Information Modeling software are mapped to the objects in the field database and are linked to the database” (Para. [0034]).
Omansky further teaches “Data structures and data sets of Building Information Model objects in the BIM Database Software Application 90 are transferred 92 to the Field Database Software Application 94. The unique data structures and data sets, related to one or may field operations, authored out in field, on the job site and at the point of construction, by construction field personnel, are then transferred 93 from the Field Database Software Application 94 back to the respective Building Information Model objects in the BIM Database Software Application 90.” (Para. [0049]) where “In the graphical user interface of the BIM Database Software Application 90, the data structures and data sets of Building Information Model objects are generally represented by visual or pictorial, three-dimensional (3D) shapes including building geometry and spatial relationships 91.” (Para. [0050]).
display the portion of the inspection data mapped to the portion of the three-dimensional model data on the display device.
Omansky teaches displaying the list of text data on a display device (Figure 5). Omansky further teaches Figure 10A as a screenshot, and thus is displaying, imported or received data structures and data sets (inspection data) which are “mapped to the objects in the field database and are linked to the database” (Paras. [0022] & [0034]). Omansky also teaches “In the graphical user interface of the BIM Database Software Application 90, the data structures and data sets of Building Information Model objects are generally represented by visual or pictorial, three-dimensional (3D) shapes including building geometry and spatial relationships 91.” (Para.[0050]) and Figure 10A also shows a three-dimensional model to which data is mapped being displayed.
Regarding Claim 6:
Liu and Omansky further teach:
Wherein the three-dimensional model data includes at least a member region of the structure and data of a member of the structure.
Omansky teaches 3d model data where “FIG. 9, which shows a representation of a user interface 120 for an example BIM database application with visualization of model objects from business rules and logic by the field database software application of the present invention” (Para. [0055]). Omansky further teaches “the Status value 121 of each unique virtual model object in the Field database” (Para. [0055]) thereby teaching a member region and data of the member.
Regarding Claim 7:
Liu and Omansky further teach:
Wherein the inspection data includes a plurality of types of data.
Omansky teaches a plurality of types of data by teaching “the invention selects and filters the data sets from the BIM Database 10 to the Field Database 13, via the Integration Adapter 12 and set of rules in XML or JSON, to include only data sets relevant to and material in value to the construction field operation, and to exclude data sets not relevant to and immaterial in value to the construction field operation. For example, in a Mechanical Commissioning field operation, the invention selects and filters only data sets related to the Mechanical Commissioning of Heating, Ventilating, and Air-Conditioning (HVAC) assemblies, systems, equipment and components, such as System Number, System Name, Equipment Number, Equipment Name, Manufacturer, Serial Number, Model Number, and Type, and not data sets related to building geometry, spatial relationships and geographic information. Different construction field operations require different data sets for proper execution and administration.” (Para. [0037]).
Regarding Claim 8:
Liu and Omansky further teach:
Wherein the plurality of types of data include
a captured image,
Omansky teaches “Exemplary documents include operations and maintenance manuals, shop drawings, test reports, construction drawings, "as-builts" or as-installed drawings, specifications, riser diagrams, P&ID piping and instrumentation diagrams, submittals, preventive maintenance plans, safety tasks, troubleshooting procedures, start-up procedures, shut-down procedures, emergency operating procedures, warranty, warranty guarantor, and spare parts lists.” (Para. [0009])
a panoramic composite image,
Omansky teaches “systems of the invention are able to deliver data structures and data sets generated in Building Information Modeling software (BIM) to construction field operations out in the field, on the job site, and at the point of construction. In FIG. 1 there is shown a BIM Level of Detail "pyramid" diagram. In the building construction phase, the invention only requires the Level of Detail with Approximate Geometry (Level of Detail 200) 31 as a minimum level of detail with approximate geometry, not precise geometry, in the Field Database for construction field operations, whereas the BIM Database requires varying levels of detail from Conceptual Level of Detail (Level of Detail 100) 30 to As-built Level of Detail (Level of Detail 500) 34 depending on the phase of the project.” (Para. [0025] & Fig. 1) thereby teaching a panoramic composite image as the “conceptual level of detail 100” when the term “panoramic composite image” is given its broadest reasonable interpretation.
damage information, and
Omansky teaches “In a Commissioning construction field operation example, the Status value 121 of each unique virtual model object in the Field database determines its color 122, 123, 124 in the BIM Database, thereby creating a virtual "heat map" or virtual "weather map" of the current status systems and equipment in the physical construction. In a Commissioning construction field operation example, if the current Status value=Functional Test, then the associated model object renders in a green color. If the current Status value=Pre-Functional Test, then the associated model object renders in a blue color. If the current Status value=Powered Up, then the associated model object renders in a red color. If the current Status value=null or no value, then the associated model object renders in a grey color.” (Para. [0055]).Omansky further teaches “exemplary documents include…emergency operating procedures, warranty, warranty guarantor, and spare parts lists” (Para. [0009]) which are all understood as information related to damage.
a two-dimensional drawing.
Omansky teaches “Exemplary documents include operations and maintenance manuals, shop drawings, test reports, construction drawings, "as-builts" or as-installed drawings, specifications, riser diagrams, P&ID piping and instrumentation diagrams, submittals, preventive maintenance plans, safety tasks, troubleshooting procedures, start-up procedures, shut-down procedures, emergency operating procedures, warranty, warranty guarantor, and spare parts lists.” (Para. [0009]).
Regarding Claim 9:
Liu and Omansky further teach wherein the processor:
Displays at least one type of data on the display device from the plurality of types of data included in the inspection.
Omansky teaches a plurality of types of data by teaching “the invention selects and filters the data sets from the BIM Database 10 to the Field Database 13, via the Integration Adapter 12 and set of rules in XML or JSON, to include only data sets relevant to and material in value to the construction field operation, and to exclude data sets not relevant to and immaterial in value to the construction field operation. For example, in a Mechanical Commissioning field operation, the invention selects and filters only data sets related to the Mechanical Commissioning of Heating, Ventilating, and Air-Conditioning (HVAC) assemblies, systems, equipment and components, such as System Number, System Name, Equipment Number, Equipment Name, Manufacturer, Serial Number, Model Number, and Type, and not data sets related to building geometry, spatial relationships and geographic information. Different construction field operations require different data sets for proper execution and administration.” (Para. [0037]).
Regarding Claim 10:
Liu and Omansky further teach:
Wherein the inspection data includes a plurality of captured images; and
Omansky teaches “Exemplary documents include operations and maintenance manuals, shop drawings, test reports, construction drawings, "as-builts" or as-installed drawings,” (Para. [0009]) thereby teaching capturing a plurality of documents as images in the form of drawings.
The processor is configured to display the captured image satisfying a condition on the display device from the plurality of captured images to be displayed.
Omansky teaches “Exemplary documents include operations and maintenance manuals, shop drawings, test reports, construction drawings, "as-builts" or as-installed drawings,” (Para. [0009]) thereby teaching capturing a plurality of documents as images in the form of drawings, where the drawings are “‘as-builts’ or as-installed drawings” thus satisfying a condition of installation/built.Omansky further teaches “data to be added/updated to the model” including properties such as installation information such as “install date” and “install complete” (Paras. [0068] – [0069]) thereby further supporting the teaching of capturing and displaying information satisfying a condition.
Regarding Claim 11:
Liu and Omansky further teaches wherein the processor is configured to:
Analyze the text data of the inspection record to extract past inspection data corresponding to the member, the position of the damage, and the type of the damage; and
Liu teaches “identifying a component of the object in the 3D virtual model that is linked to the linked text, and highlighting, in response to identifying the component of the object in the 3D virtual model, the component of the object in the 3D virtual model displayed on display interface of the display device.” (Abstract).
Omansky teaches “transmitting the data structures and data sets includes transmitting data at a level of detail required for the field operations or processes” (Claim 5) and “The invention selects and filters the data structures of building assemblies, systems, equipment and components from the BIM Database 10 to the Field Database 13, via the Integration Adapter 12 and set of rules in XML or JSON, to include only data structures relevant to and material in value to the construction field operation, and to exclude data structures not relevant to and immaterial in value to the construction field operation” (Para. [0036]).Omansky further teaches past inspection data by teaching “associate documents or electronic links to documents with the selected data structures and data sets. Exemplary documents include…test reports” (Para. [0009]) and “the data structures and data sets relate to building assemblies, materials, systems, sub-systems, equipment, components, and inter-relations thereof in a building project. In other embodiments, the data structures and data sets relate to…issue tracking, field reports, materials tracking, safety, quality assurance, quality control, work lists, punch lists, and handover.” (Para. [0010]).
Display the past inspection data on the display device.
Omansky teaches “transmitting the data structures and data sets includes transmitting data at a level of detail required for the field operations or processes” (Claim 5) where “FIG. 5 is a drawing showing a representation of graphical user interfaces and data displays of an example BIM database application and of the field database software application of the present invention” (Para. [0017]) and Figs. 5, 6, and 9 depict displaying corresponding portions of the 3d model data and/or a screen of the extracted/relevant inspection data “at a level of detail required for the field operations or processes”.
Regarding Claim 14:
Liu and Omansky further teach:
Wherein the inspection record data is created in the predetermined format in which the text data of information of the member of the structure, the damage to the structure, the position of the damage, and findings of the damage is input for each of the plurality of inspection points.
Omansky teaches the data being created in a predetermined format by teaching “In the graphical user interface (QUI) of the Field Database Software Application 94, the unique data structures and data sets, related to one or may field operations, and associated with respective Building Information Model objects are generally represented in a tabular format with text values in columns and rows 95 or a list view. Building Information Model data structures and data sets are translated from shapes 91 to text 95, retaining the hierarchical relationships of the Building Information Model. Then, text and text-based values 95 are transferred back 93 to the associated objects in the Building Information Model.” (Para. [0050]) where “The tabular format with text values in columns and rows 95 facilitates use by construction field personnel and generally lowers barriers to adoption. The tabular format is faster and easier to navigate, search and read through a large amount of information in a list format, filtered, grouped and sorted as required, than in a three-dimensional (3D) shapes format.” (Para. [0052]).
Claim(s) 16 is rejected under 35 U.S.C. 103 as being unpatentable over Omansky and Liu, and further in view of (U.S. Pre-Grant Publication No. 2018/0025659).
Regarding Claim 16:
Some of the limitations herein are similar to some or all of the limitations of Independent Claim 15.
Liu and Omansky explicitly teach all of the elements of the claimed invention as recited above except:
Analyze the text data to determine whether the text data include text data related to progressiveness of damage; and
In response to determining that the text data include the text data related to the progressiveness of the damage, extract past inspection data corresponding to the at least one of the plurality of inspection points, and display the past inspection data on the display device.
However, in the related field of endeavor of inspection result retrieval, Yamagishi teaches:
Analyze the text data to determine whether the text data include text data related to progressiveness of damage; and
Yamagishi teaches “Accordingly, changes over time in an inspection portion of the construction 9 and changes over time in the damage D that has occurred in the inspection portion can be checked.” (Para. [0109]).
In response to determining that the text data include the text data related to the progressiveness of the damage, extract past inspection data corresponding to the at least one of the plurality of inspection points, and display the past inspection data on the display device.
Yamagishi teaches “the inspection result retrieval device 10A according to the second embodiment performs image processing for matching the shape and position of the construction 9 in the inspection image data 27 included in the first inspection result 13 and those of the construction 9 in the inspection image data 27 included in each of the second inspection results 17, the first inspection result 13 and the second inspection results 17 being the results of inspection of the same construction 9, and thereafter, causes the display unit 20 to perform display so as to enable a comparison.” (Para. [0109]).Yamagishi further teaches “in a case where a user selects the second inspection result 17 (including the specific inspection result 17A in the fifth embodiment) that is a desired one on the retrieval result display screen 38 (see FIG. 7) displayed on the display unit 20, the inspection history of the construction 9 (hereinafter referred to as the specific construction 9) that is the inspection target in the selected second inspection result 17 is displayed on the display unit 20.” (Para. [0129]).
Thus, it would have been obvious to one of ordinary skill in the art, having the teachings of Yamagishi, Liu, and Omansky at the time that the claimed invention was effectively filed, to have modified the highlighting feature of a particular component when selecting a particular piece of text, as taught by Liu, and the systems and methods for construction field management and operations with building information modeling, as taught by Omansky, with the tracking of changes in the damage of the construction over time in an inspection portion of the construction, as taught by Yamagishi.
One would have been motivated to make such modification because Yamagishi teaches tracking damage history and predicting the development of damage such that “ the severity of the damage D that has occurred in the construction 9 can be determined, and furthermore, the time when the damage D of the construction 9 needs to be repaired can be appropriately determined.” (Para. [0093]) and it would be obvious to a person having ordinary skill in the art that knowing the time when damaged components need to be repaired would improve the ability to manage the structure and proactively schedule repairs as well as order any necessary replacement parts in advance.
Response to Amendment
Applicant’s Amendments, filed on 9/25/2025, are acknowledged and accepted.
In light of the Amendments filed on 9/25/2025, the claim objection to claims 1-2, 12, and 13 has been withdrawn.
Response to Arguments
On page 7 of the Remarks filed on 9/25/2025, Applicant argues with respect to the 101 rejection that “claim 1 recites to "analyze the text data of the inspection record to specify a member of the structure, a position of damage to the structure, and a type of the damage to the structure corresponding to the text data" and "extract a portion of the three-dimensional model data corresponding to the member, the position of the damage, and the type of the damage." Amended claim 1 does not, however, recite any "observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper"” and thus “Applicant respectfully submits that amended claim 1 is not directed to an abstract idea”.Applicant’s statement that “claim 1 does not, however, recite any "observation, evaluation, judgement, and/or opinion capable of being performed by the human mind, or by a human using a pen and paper"” is not agreed upon and the amended limitations are addressed further in the rejection above.
On page 8 of the Remarks filed on 9/25/2025, Applicant argues with respect to the 101 rejection that “amended claim 1 is patent-eligible because amended claim 1 recites the combination to "analyze the text data of the inspection record to specify a member of the structure, a position of damage to the structure, and a type of the damage to the structure corresponding to the text data" and "extract a portion of the three-dimensional model data corresponding to the member, the position of the damage, and the type of the damage." Applicant respectfully submits that the artisan of ordinary skill should have understood that specifying a member of the structure, a position of damage to the structure, and a type of the damage to the structure corresponding to the text data would have required a method of textual analysis, for example, such as named entity recognition (NER) (see Specification at 60). Accordingly, amended claim 1 recites limitations that reflect a derived association between unstructured text data of an "inspection record data created in a predetermined format in which text data of predetermined information is input for each of a plurality of inspection points of the structure" to "three-dimensional model data of the structure," which is reflected as "a portion of the three-dimensional model data corresponding to the member, the position of the damage, and the type of the damage.”".Applicant’s argument is not convincing because the argued action that an artisan of ordinary skill would have understood such as specifying a member of the structure, a position of damage to the structure, and a type of the damage to the structure corresponding to the text data , using textual analysis such as named entity recognition, is something that the artisan or ordinary skill in the art could have also performed. It is further not agreed that the combination of limitations recited in the amended claim 1 integrates the claims into a practical application.
On page 9 of the Remarks filed on 9/25/2025, Applicant argues with respect to the 101 rejection that “claim 1 reflects an improvement to the technical field of infrastructure inspection. "With viewing of a damage diagram created in a predetermined format, even an expert who is different from the inspector who actually performs the inspection can grasp a progressing situation of damage to the structure and formulate a maintenance plan for the structure." Specification at [paragraph] 3. "By the way, in a case where an inspection record is checked, there may be a case where it is desired to refer to damage fact information (captured image, drawing, three- dimensional model data, or the like) related to a damage situation from text information of the inspection record. However, the damage fact information is not associated with the inspection record created in the predetermined format, and thus there is a problem that it takes time and effort to refer to the damage fact information." Specification at [paragraph] 6. Thus, the Specification discloses the technical problem of the disassociation between damage information and an inspection report, which increases the difficulty of reviewing the damage to a structure.” and “The Specification discloses a technical solution to the technical problem by extracting "corresponding portion on the three-dimensional model data 101 and/or the inspection data 103, which are corresponding to this text data, based on the information of the member, the position, and the damage." Specification at 56. Accordingly, Applicant respectfully submits that it should have been apparent to the artisan of ordinary skill that the Specification describes an improvement to the technology of infrastructure inspection, and the claims reflect the particular way in which the technical solution to the technical problem of disassociation between damage information and an inspection report.”Applicant’s argument is not convincing because associating disassociated damage information with an inspection report (the problem being solved) is not understood as a technical problem in technology, but instead is something capable of being performed by the human reviewing the damage information and the inspection report (observation and evaluation) and making associations (judgement and/or opinion). Upon further review of the amended claims, the additional elements were not found to recite significantly more than the recited judicial exception (Step 2B) as is further explained in the rejection above.
On page 11 of the Remarks filed on 9/25/2025, Applicant argues with respect to the prior art rejection that “Liu fails to disclose, suggest, or otherwise render obvious to "analyze the text data of the inspection record to specify a member of the structure, a position of damage to the structure, and a type of the damage to the structure corresponding to the text data" and "extract a portion of the three-dimensional model data corresponding to the member, the position of the damage, and the type of the damage."” because “Instead, Liu describes "a user input selecting a linked text displayed in the textual instruction section, identifying a component of the object in the 3D virtual model that is linked to the linked text." Liu at Abstract. Accordingly, Liu at most describes "linked texts 126, 128, 130 that are linked with components in the virtual model section." Liu at [paragraph] 19.”Applicant’s argument and the related amended limitations has been given further time and consideration but were not found to overcome the combination of the Liu and Omansky references, as is further addressed in the rejection of the amended claims above.
On page 7 of the Remarks filed on 9/25/2025, Applicant argues with respect to the prior art rejection that “the artisan of ordinary skill should not have understood the conventional linking between text and objects in Liu to have been analogous to analyzing "the text data of the inspection record to specify a member of the structure, a position of damage to the structure, and a type of the damage to the structure corresponding to the text data" and extracting "a portion of the three-dimensional model data corresponding to the member, the position of the damage, and the type of the damage," as recited in amended claim 1.” because “there is no disclosure or suggestion that the linked texts 126, 128, 130 in Liu are analyzed in any way to specify any portion of the component of the object 106 in Liu. Indeed, Applicant respectfully submits that the artisan of ordinary skill should have understood that no textual analysis is necessary in Liu because of the stored association between the linked texts 126, 128, 130 and the components 110, 112, 114 of the object 106 in Liu.”
Applicant’s argument and the related amended limitations has been given further time and consideration but were not found to overcome the combination of the Liu and Omansky references, as is further addressed in the rejection of the amended claims above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bussey et al. (U.S. Patent No. 8,319,792) teaches composing a computer-aided design (CAD) drawing that includes modeled components and non-modeled components. The method includes receiving a selection of properties for a non-modeled component to include in the CAD drawing, where the non-modeled component defines an element of an object modeled in the CAD drawing that is not included in a display representation of the object including an instance of the non-modeled component in the CAD drawing. Although not shown in display representations, the non-modeled elements may be displayed in a browse view showing the elements included in a CAD drawing, and in a bill of materials generated from the CAD drawing.The reference further teaches “Illustratively, 3D model 120 includes 3D model parts 122, virtual components 124, and part associations 126. 3D model parts 122 are elements of a mechanical design that are displayed in a 3D model view 112. Virtual components 124 are elements of the mechanical design represented by 3D model 120 that are not displayed in 3D model view 112. Virtual components 124 may; however, be included in displays generated by a model browser tool 114 and in BOMs generated using a BOM tool 116. Part associations 126 enable 3D model parts 122 to be associated with an associative virtual component, as appropriate in a given case. In other words, part associations 126 specify what virtual components 124 are associated with a given 3D model part. Further, virtual components 122 may have properties that describe aspects of the real-world object modeled by the virtual component. For example, properties such as mass and volume may be included in the properties of a virtual component. Such proprieties may be useful for engineering calculations of the expected weight, center of gravity, mass, volume and other physical properties of the real-world object modeled by 3D model 120.” (Col. 4 Lines 12-32).
Basile et al. (U.S. Patent No. 6,725,097) teaches assisting an operator in surveying defects on a building site which consists in providing him with a portable terminal displaying images of the works and inputting data concerning his observations, under the control of a software operating with a database of the works, comprising files defining the images displayed on the terminal graphic interface, data structures relating respectively to types of defects liable to occur in the works, and if any, descriptive data of defects previously observed on the works, organized in accordance with the associated data structures. In response to the selection of a type of fault by the operator, said software activates a dialogue, by means of the associated data structure, to input parameters describing said defects, which are used to constitute a representation of the defect on the image displayed on the graphic interface and to update the data describing the defects.
Mildrew et al. (U.S. Patent No. 10,139,985) teaches defining, displaying and interacting with tags in a 3D model. In an embodiment, a method includes generating, by a system including a processor, a three-dimensional model of an environment based on sets of aligned three-dimensional data captured from the environment, and associating tags with defined locations of the three-dimensional model, wherein the tags are respectively represented by tag icons that are spatially aligned with the defined locations of the three-dimensional model as included in different representations of the three-dimensional model rendered via an interface of a device, wherein the different representations correspond to different perspectives of the three-dimensional model, and wherein selection of the tag icons causes the tags respectively associated therewith to be rendered at the device.The reference further teaches “a tag entry for a point of interest in a 3D model may provide any amount of textual information associated with the point of interest, such as title, byline, description, details, summary or report. In another example, a tag associated with a point of interest can identify any number of images associated with the point of interest, including photorealistic and panorama images. In another example, a tag associated with a point of interest can include audio associated with the point of interest, a video associated the point of interest, 360° panorama associated with the point of interest, or a combination of image, video and audio associated with the point of interest. In another example, a tag associated with a point of interest can include a hyperlink to external websites, as indicated by a uniform resource locator (URL) or uniform resource identifier (URI)” (Col. 22 Lines 24-38).
Stanic (U.S. Pre-Grant Publication No. 2015/0186472) teaches an automated process collects and organizes field data from an inspection of a building or other structure such as pipe supports, bridges, buildings, over head supports, and smoke stacks.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT F MAY whose telephone number is (571)272-3195. The examiner can normally be reached Monday-Friday 9:30am to 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached at 571-270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROBERT F MAY/Examiner, Art Unit 2154 12/27/2025
/SYED H HASAN/Primary Examiner, Art Unit 2154