DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
All Information Disclosure Statements filed on or before 07/17/2025 have been considered by examiner.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 10 are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Fisher (US 20210065356 A1).
With respect to claim 1, Fisher teaches a method for monitoring a status of a shell and tube device during a projection-type maintenance activity (“Robot 120 may be utilized with its associated camera 124 and end effector 122/tool 123 to perform an inspection of heat exchanger 100 (FIG. 1), including automatic inspection by and guidance of the robot, by tracking the robot's location through the heat exchanger based on image data acquired by the camera as the robot moves rather than upon aggregation of sequential robot movements.” Paragraph 0051), said method comprising: acquiring an initial digital image (Di) of a tube end on at least a portion of a tube sheet of the shell and tube device (“Processing circuitry 50 (FIG. 14) translates data detected in images acquired by the robot camera into tube sheet space, so that positions of tubes that are within each acquired image are identifiable in tube sheet space and stored in memory” paragraph 0053 and figure 15 and “The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image…” paragraph 0038);
acquiring a later digital image (Dx) of the tube end at a later time during the projection- type maintenance activity (“The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image to predetermined data locating the characteristic on the tube sheet, to thereby determine the image's current location with respect to the tube sheet map and to identify other tube sheet characteristics in the image with respect to the tube sheet map. The processing circuitry may compare an unknown heat exchanger characteristic, such as a tube location, in a given image to a known heat exchanger characteristic in a prior image to determine or confirm the identity of the heat exchanger characteristic in the present image.” Paragraph 0038);
and comparing the initial digital image (Di) with the later digital image (Dx) to identify a presence of an object projecting from the tube end (“The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image to predetermined data locating the characteristic on the tube sheet, to thereby determine the image's current location with respect to the tube sheet map and to identify other tube sheet characteristics in the image with respect to the tube sheet map. The processing circuitry may compare an unknown heat exchanger characteristic, such as a tube location, in a given image to a known heat exchanger characteristic in a prior image to determine or confirm the identity of the heat exchanger characteristic in the present image.” Paragraph 0038).
With respect to claim 10, Fisher teaches the method of claim 1,wherein said shell and tube device is one of a reactor (“The discussion below, with reference to the steps provided at FIG. 15, the image provided at FIG. 5, and the user interface display illustrations of FIGS. 6A and 6B, provides an example of a method for tracking a robot on a surface such as a tube sheet of a nuclear reactor.” Paragraph 0054).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Fisher (US 20210065356 A1) in view of Karkoub (Karkoub M, Bouhali O, Sheharyar A. Gas Pipeline Inspection Using Autonomous Robots With Omni-Directional Cameras. IEEE. 10 December 2020).
With respect to claim 2, Fisher teaches the method of claim 1. Fisher does not teach wherein each acquiring step comprises acquiring one pair of digital images of the tube end from two different vantage points.
Karkoub teaches acquiring one pair of digital images of the tube end from two different vantage points (see fig.6, fig. 16, and figs.24-25).
Karkoub is analogous art in the same field of endeavor as the claimed invention. Karkoub is directed towards tube maintenance (see fig. 1). A person of ordinary skill before the effective filing date of the claimed invention would have found it obvious to combine Fisher and Karkoub by utilizing Karkoub’s catadioptric camera on its Fisher’s robots. With the expectation that doing so would lead to a more comprehensive camera view of the tube (see fig.6, fig. 16, and figs.24-25).
With respect to claim 3, Fisher teaches the method of claim 1, but does not teach wherein acquiring said pair of digital images of the tube end from two different viewing angles (see fig.6, fig. 16, and figs.24-25) is performed using a catadioptric system comprising a single imaging device.
Karkoub teaches wherein acquiring said pair of digital images of the tube end from two different viewing angles is performed using a catadioptric system comprising a single imaging device (“The vision system considered in this work, called catadioptric omni-directional vision system, which is a combination of a reflective mirror and a conventional dioptric camera” page 3 paragraph 1).
Karkoub is analogous art in the same field of endeavor as the claimed invention. Karkoub is directed towards tube maintenance (see fig. 1). A person of ordinary skill before the effective filing date of the claimed invention would have found it obvious to combine Fisher and Karkoub by utilizing Karkoub’s catadioptric camera on its Fisher’s robots. With the expectation that doing so would lead to a more comprehensive camera view of the tube (see fig.6, fig. 16, and figs.24-25).
Claims 4-5 is rejected under 35 U.S.C. 103 as being unpatentable over Fisher as applied to claim 1 above, and further in view of Moselhi (US 20030023404 A1).
With respect to claim 4, Fisher teaches the method of claim 1, but does not teach it further comprising the step of creating a data record in a relational database for the tube end, said data record containing a value representing the presence of the identified object.
Moselhi teaches the step of creating a data record in a relational database (“There are different database models, of which the relational model is the most commonly used in engineering applications” paragraph 0308) product containing a defect (“To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” paragraph 0306 and fig.94 ).
Moselhi is analogous art in the same field of endeavor as the claimed invention. Moselhi is directed towards a database for repair operations (“To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” paragraph 0306 and fig.94 ). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the maintenance procedure of Fisher with the relational database of Moselhi by utilizing Moselhi’s extended image processing capabilities and storage organization scheme, with the expectation that doing so would improve repair operations (“Automating these processes could also provide an incentive for assessing sewer networks more regularly as a part of preventive maintenance programs” paragraph 0024).
With respect to claim 5, Fisher teaches the method of claim 1, but does not teach it further comprising the step of creating a data record in a relational database for a plurality of tube ends of the shell and tube device.
Moselhi teaches the step of creating a data record in a relational database for a plurality of products (“To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” and “There are different database models, of which the relational model is the most commonly used in engineering applications” paragraph 0308)
Moselhi is analogous art in the same field of endeavor as the claimed invention. Moselhi is directed towards a database for repair operations (“To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” paragraph 0306 and fig.94 ). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the maintenance procedure of Fisher with the relational database of Moselhi by utilizing Moselhi’s extended image processing capabilities and storage organization scheme, with the expectation that doing so would improve repair operations (“Automating these processes could also provide an incentive for assessing sewer networks more regularly as a part of preventive maintenance programs” paragraph 0024).
Claims 6 and 9 are rejected under 35 U.S.C. 103 as being unpatentable over Fisher and Moselhi as applied to claim 4 above, and further in view of Dragan (Export SQL Server Data to Multiple Excel Worksheets by Mircea Dragan. November 18, 2020).
With respect to claim 6, Fisher and Moselhi teach the method of claim 4 but do not teach it further comprising the step of using one or more data records stored in the relational database to produce one or more of tables, graphs, spreadsheets, and color-coded summary graphics.
Dragan teaches converting relational databases into spreadsheets and tables (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1).
Dragan is analogous art reasonably pertinent to the problem of data storage and processing. Dragan is directed towards a methodology of converting relational databases into spreadsheets and tables (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Fisher and Moselhi by utilizing Dragan’s teachings of data conversion, with the expectation that doing so would lead to the ability to analyze data present in the relational database (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1).
With respect to claim 9, Fisher and Moselhi teach the method of claim 4. Fisher additionally teaches measuring one or more workspace parameters (“Processing circuitry 50 (FIG. 14) translates data detected in images acquired by the robot camera into tube sheet space, so that positions of tubes that are within each acquired image are identifiable in tube sheet space and stored in memory” paragraph 0053 and figure 15 and “The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image…” paragraph 0038), but does not teach recording workspace parameter measurements in the relational database, and optionally, presenting said workspace parameter measurements on a visual display.
Moselhi teaches recording workspace parameter measurements in the relational database (“To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” and “There are different database models, of which the relational model is the most commonly used in engineering applications” paragraph 0308).
Dragan teaches optionally, presenting said workspace parameter measurements on a visual display (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1).
Dragan is analogous art reasonably pertinent to the problem of data storage and processing. Dragan is directed towards a methodology of converting relational databases into spreadsheets and tables (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Fisher and Moselhi by utilizing Dragan’s teachings of data conversion, with the expectation that doing so would lead to the ability to analyze data present in the relational database (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1).
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Fisher as applied to claim 1 above, and further in view of Thompson (US 20200250810 A1).
With respect to claim 7, Fisher teaches the method of claim 1, but does not teach it further comprising the step of producing performance metrics for the projection-type maintenance activity, wherein the producing step comprises calculating and displaying the performance metrics in a table, graph, spreadsheet, or color-coded summary graphic.
Thompson teaches the step of producing performance metrics for the projection-type maintenance activity (“It is the object of many embodiments to provide an improved method of visual reporting which may include an interactive screen so that while recording, a user may be able to advise of a pass, fail and/or comment section as it relates to being able to input data preferably by videotaping and/or capturing still images simultaneously, possibly by taking an image upon interacting with the display screen.” Paragraph 0007), wherein the producing step comprises calculating and displaying the performance metrics in a table (see fig. 7)
Thompson is analogous art in the same field of endeavor as the claimed invention. Thompson is directed toward inspection reporting (“It is the object of many embodiments to provide an improved method of visual reporting which may include an interactive screen so that while recording, a user may be able to advise of a pass, fail and/or comment section as it relates to being able to input data preferably by videotaping and/or capturing still images simultaneously, possibly by taking an image upon interacting with the display screen.” Paragraph 0007). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Fisher and Thompson by utilizing Thompsons report visualization methodology to show Fisher’s maintenance procedural findings, with the expectation that doing so would lead to improved reporting of maintenance findings (“It is the object of many embodiments to provide an improved reporting system which preferably simultaneously provides a text (and/or graphic) overlay over a display set to capture video so that what the user is advised of specific steps to take or questions to answer may be displayed on the display while taking the video of the information to be obtained, possibly providing an audio read graphic overlay for some applications” paragraph 0006).
With respect to claim 8, Fisher and Thompson teach the method of claim 7. Thompson further comprising displaying said performance metrics on a mobile visual display device (see fig. 7).
Claims 11-13, 16-17, and 19-22 are rejected under 35 U.S.C. 103 as being unpatentable over Fisher in view of Karkoub, Moselhi, and Thompson.
With respect to claim 11, Fisher teaches for a shell and tube device comprising a tube sheet including a plurality of tube ends arranged in a fixed pattern of rows and columns (see figure 9), a method for monitoring the status of the shell and tube device during a projection-type maintenance activity (“Robot 120 may be utilized with its associated camera 124 and end effector 122/tool 123 to perform an inspection of heat exchanger 100 (FIG. 1), including automatic inspection by and guidance of the robot, by tracking the robot's location through the heat exchanger based on image data acquired by the camera as the robot moves rather than upon aggregation of sequential robot movements.” Paragraph 0051), said method comprising: determining an initial state of an attribute (Ai) having at least two states for each of the tube ends within said pair of initial digital images (Dai and Dbi) (“Processing circuitry 50 (FIG. 14) translates data detected in images acquired by the robot camera into tube sheet space, so that positions of tubes that are within each acquired image are identifiable in tube sheet space and stored in memory” paragraph 0053 and figure 15 and “The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image…” paragraph 0038),
Fisher does not teach an initial acquisition time (Ti), acquiring one pair of initial digital images (Dai and Dbi) of at least a portion of the tube sheet from two different vantage points and assigning a unique identifier to each of the tube ends, creating an initial data record in a relational database for each tube end within said pair of initial digital images (Dai and Dbi), said initial data record including: i. the initial acquisition time (Ti), ii. the unique identifier for the tube end, and iii. an initial state of the image attribute (Ai) at the initial acquisition time (Ti)
Karkoub teaches at an initial acquisition time (Ti), acquiring one pair of initial digital images (Dai and Dbi) of at least a portion of the tube sheet from two different vantage points (see fig.6, fig. 16, and figs.24-25).
Karkoub is analogous art in the same field of endeavor as the claimed invention. Karkoub is directed towards tube maintenance (see fig. 1). A person of ordinary skill before the effective filing date of the claimed invention would have found it obvious to combine Fisher and Karkoub by utilizing Karkoub’s catadioptric camera on its Fisher’s robots. With the expectation that doing so would lead to a more comprehensive camera view of the tube (see fig.6, fig. 16, and figs.24-25).
Moselhi teaches assigning a unique identifier to each product, creating an initial data record in a relational database for each product within said pair of initial digital images (Dai and Dbi) (“To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” and “There are different database models, of which the relational model is the most commonly used in engineering applications” paragraph 0308), said initial data record including: ii. the unique identifier for each product (see figure 94)
Moselhi is analogous art in the same field of endeavor as the claimed invention. Moselhi is directed towards a database for repair operations (“To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” paragraph 0306 and fig.94 ). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the maintenance procedure of Fisher and Karkoub with the relational database of Moselhi by utilizing Moselhi’s extended image processing capabilities and storage organization scheme, with the expectation that doing so would improve repair operations (“Automating these processes could also provide an incentive for assessing sewer networks more regularly as a part of preventive maintenance programs” paragraph 0024).
Thompson teaches a data record comprising the initial acquisition time (Ti) (see figure 7), and an initial state of the image attribute (Ai) at the initial acquisition time (Ti) (see figure 7).
Thompson is analogous art in the same field of endeavor as the claimed invention. Thompson is directed toward inspection reporting (“It is the object of many embodiments to provide an improved method of visual reporting which may include an interactive screen so that while recording, a user may be able to advise of a pass, fail and/or comment section as it relates to being able to input data preferably by videotaping and/or capturing still images simultaneously, possibly by taking an image upon interacting with the display screen.” Paragraph 0007). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Fisher , Karkoub, and Moselhi with Thompson by utilizing Thompsons report visualization methodology to show Fisher’s maintenance procedural findings, with the expectation that doing so would lead to improved reporting of maintenance findings (“It is the object of many embodiments to provide an improved reporting system which preferably simultaneously provides a text (and/or graphic) overlay over a display set to capture video so that what the user is advised of specific steps to take or questions to answer may be displayed on the display while taking the video of the information to be obtained, possibly providing an audio read graphic overlay for some applications” paragraph 0006).
With respect to claim 12, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 11. Fisher further teaches it further comprising: at a later acquisition time (Tx), wherein Tx > Ti, acquiring a later pair of digital images (Dax and Dbx) of at least a portion of the tube sheet (“The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image to predetermined data locating the characteristic on the tube sheet, to thereby determine the image's current location with respect to the tube sheet map and to identify other tube sheet characteristics in the image with respect to the tube sheet map. The processing circuitry may compare an unknown heat exchanger characteristic, such as a tube location, in a given image to a known heat exchanger characteristic in a prior image to determine or confirm the identity of the heat exchanger characteristic in the present image.” Paragraph 0038) and determining a later state of the attribute (Ax) for each tube end in said later pair of digital images (Dax and Dbx) (“Processing circuitry 50 (FIG. 14) translates data detected in images acquired by the robot camera into tube sheet space, so that positions of tubes that are within each acquired image are identifiable in tube sheet space and stored in memory” paragraph 0053 and figure 15 and “The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image…” paragraph 0038). Fisher additionally teaches repeating the steps of the maintenance procedure (see fig. 15)
Fisher does not teach acquiring images from two different vantage points and g) creating a later data record in the relational database for each tube end within said later pair of digital images (Dax and Dbx), said later data record including: i. the later acquisition time (Tx), ii. the unique identifier for the tube end, iii. the later state of the selected attribute (Ax) at the later acquisition time (Tx),
Karkoub teaches acquiring images from two different vantage points (see fig.6, fig. 16, and figs.24-25).
Moselhi teaches creating a later data record in the relational database for each product within said later pair of digital images (Dax and Dbx) (see fig.94), and repeatedly adding information to the database until all relevant data is added (see fig.94), and the unique identifier for the product (see fig.94)
Thompson teaches a data record including an acquisition time (see figure 7), iii. the later state of the selected attribute (Ax) at the later acquisition time (Tx) (see figure 7),
With respect to claim 13, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 11 , Fisher further teaches wherein said shell and tube device is one of a reactor (“The discussion below, with reference to the steps provided at FIG. 15, the image provided at FIG. 5, and the user interface display illustrations of FIGS. 6A and 6B, provides an example of a method for tracking a robot on a surface such as a tube sheet of a nuclear reactor.” Paragraph 0054)
With respect to claim 16, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 11. Thompson further teaches the step of producing performance metrics for the projection-type maintenance activity (“It is the object of many embodiments to provide an improved method of visual reporting which may include an interactive screen so that while recording, a user may be able to advise of a pass, fail and/or comment section as it relates to being able to input data preferably by videotaping and/or capturing still images simultaneously, possibly by taking an image upon interacting with the display screen.” Paragraph 0007), wherein the producing step comprises calculating and displaying the performance metrics in a table (see fig. 7)
With respect to claim 17, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 16. Thompson further teaches displaying said performance metrics on a mobile visual display device (see fig. 7).
With respect to claim 19, Fisher teaches for a shell and tube device comprising a tube sheet including a plurality of tube ends arranged in a fixed pattern of rows and columns (see figure 9), a method for monitoring the status of the shell and tube device during a projection-type maintenance activity (“Robot 120 may be utilized with its associated camera 124 and end effector 122/tool 123 to perform an inspection of heat exchanger 100 (FIG. 1), including automatic inspection by and guidance of the robot, by tracking the robot's location through the heat exchanger based on image data acquired by the camera as the robot moves rather than upon aggregation of sequential robot movements.” Paragraph 0051), said method comprising: b) at an initial acquisition time (Ti), acquiring at least one pair of initial digital images (Dai and Dbi) (“Processing circuitry 50 (FIG. 14) translates data detected in images acquired by the robot camera into tube sheet space, so that positions of tubes that are within each acquired image are identifiable in tube sheet space and stored in memory” paragraph 0053 and figure 15 and “The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image…” paragraph 0038) and determining an initial state of an attribute (Ai) having at least two states for each of the tube ends within said initial Enhanced digital image (Ei) (“Processing circuitry 50 (FIG. 14) translates data detected in images acquired by the robot camera into tube sheet space, so that positions of tubes that are within each acquired image are identifiable in tube sheet space and stored in memory” paragraph 0053 and figure 15 and “The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image…” paragraph 0038)
Fisher does not teach an enhanced image, an initial acquisition time (Ti), acquiring one pair of initial digital images (Dai and Dbi) of at least a portion of the tube sheet from two different vantage points and assigning a unique identifier to each of the tube ends, creating an initial data record in a relational database for each tube end within said pair of initial digital images (Dai and Dbi), said initial data record including: i. the initial acquisition time (Ti), ii. the unique identifier for the tube end, and iii. an initial state of the image attribute (Ai) at the initial acquisition time (Ti)
Karkoub teaches at an initial acquisition time (Ti), acquiring at least one pair of initial digital images (Dai and Dbi) of at least a portion of the tube sheet from two different vantage points (see fig.6, fig. 16, and figs.24-25)
Karkoub is analogous art in the same field of endeavor as the claimed invention. Karkoub is directed towards tube maintenance (see fig. 1). A person of ordinary skill before the effective filing date of the claimed invention would have found it obvious to combine Fisher and Karkoub by utilizing Karkoub’s catadioptric camera on its Fisher’s robots. With the expectation that doing so would lead to a more comprehensive camera view of the tube (see fig.6, fig. 16, and figs.24-25).
Moselhi teaches assigning a unique identifier to each product (see figure 94), processing the at least one pair of initial digital images to create an initial Enhanced Digital Image (Ei) (“Various image analysis techniques were tried and analyzed, aiming to enhance the image of defects and isolate them from background” paragraph 0242), and creating an initial data record in a relational database for each product within said initial Enhanced digital image (Ei) “To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” and “There are different database models, of which the relational model is the most commonly used in engineering applications” paragraph 0308, said initial data record including: the unique identifier for the products (see figure 94)
Moselhi is analogous art in the same field of endeavor as the claimed invention. Moselhi is directed towards a database for repair operations (“To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” paragraph 0306 and fig.94 ). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the maintenance procedure of Fisher and Karkoub with the relational database of Moselhi by utilizing Moselhi’s extended image processing capabilities and storage organization scheme, with the expectation that doing so would improve repair operations (“Automating these processes could also provide an incentive for assessing sewer networks more regularly as a part of preventive maintenance programs” paragraph 0024).
Thompson teaches a data record comprising the initial acquisition time (Ti) (see figure 7), and an initial state of the image attribute (Ai) at the initial acquisition time (Ti) (see figure 7).
Thompson is analogous art in the same field of endeavor as the claimed invention. Thompson is directed toward inspection reporting (“It is the object of many embodiments to provide an improved method of visual reporting which may include an interactive screen so that while recording, a user may be able to advise of a pass, fail and/or comment section as it relates to being able to input data preferably by videotaping and/or capturing still images simultaneously, possibly by taking an image upon interacting with the display screen.” Paragraph 0007). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Fisher , Karkoub, and Moselhi with Thompson by utilizing Thompsons report visualization methodology to show Fisher’s maintenance procedural findings, with the expectation that doing so would lead to improved reporting of maintenance findings (“It is the object of many embodiments to provide an improved reporting system which preferably simultaneously provides a text (and/or graphic) overlay over a display set to capture video so that what the user is advised of specific steps to take or questions to answer may be displayed on the display while taking the video of the information to be obtained, possibly providing an audio read graphic overlay for some applications” paragraph 0006).
With respect to claim 20, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 19, Fisher further teaches it further comprising: at a later acquisition time (Tx), wherein Tx > Ti, acquiring a later pair of digital images (Dax and Dbx) of at least a portion of the tube sheet (“The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image to predetermined data locating the characteristic on the tube sheet, to thereby determine the image's current location with respect to the tube sheet map and to identify other tube sheet characteristics in the image with respect to the tube sheet map. The processing circuitry may compare an unknown heat exchanger characteristic, such as a tube location, in a given image to a known heat exchanger characteristic in a prior image to determine or confirm the identity of the heat exchanger characteristic in the present image.” Paragraph 0038) and determining a later state of an attribute (Ax) for each tube end (“Processing circuitry 50 (FIG. 14) translates data detected in images acquired by the robot camera into tube sheet space, so that positions of tubes that are within each acquired image are identifiable in tube sheet space and stored in memory” paragraph 0053 and figure 15 and “The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image…” paragraph 0038). Fisher additionally teaches repeating the steps of the maintenance procedure (see fig. 15)
Fisher does not teach an enhanced image, acquiring images from two different vantage points and g) creating a later data record in the relational database for each tube end within said later pair of digital images (Dax and Dbx), said later data record including: i. the later acquisition time (Tx), ii. the unique identifier for the tube end, iii. the later state of the selected attribute (Ax) at the later acquisition time (Tx),
Karkoub teaches acquiring images from two different vantage points (see fig.6, fig. 16, and figs.24-25).
Moselhi teaches processing the at least one pair of later digital images to create a later Enhanced Digital Image (Ex) (“Various image analysis techniques were tried and analyzed, aiming to enhance the image of defects and isolate them from background” paragraph 0242), creating a later data record in the relational database for each product within said later Enhanced digital image (Ex) (see fig.94), and repeatedly adding information to the database until all relevant data is added (see fig.94), and the unique identifier for the tube end (see fig.94) said later data record including:
Thompson teaches a data record including an acquisition time (see figure 7), iii. the later state of the selected attribute (Ax) at the later acquisition time (Tx) (see figure 7),
With respect to claim 21, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 19. Fisher further teaches wherein said shell and tube device is one of a reactor (“The discussion below, with reference to the steps provided at FIG. 15, the image provided at FIG. 5, and the user interface display illustrations of FIGS. 6A and 6B, provides an example of a method for tracking a robot on a surface such as a tube sheet of a nuclear reactor.” Paragraph 0054)
With respect to claim 22, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 19. Thompson further teaches the step of producing performance metrics for the projection-type maintenance activity (“It is the object of many embodiments to provide an improved method of visual reporting which may include an interactive screen so that while recording, a user may be able to advise of a pass, fail and/or comment section as it relates to being able to input data preferably by videotaping and/or capturing still images simultaneously, possibly by taking an image upon interacting with the display screen.” Paragraph 0007), wherein the producing step comprises calculating and displaying the performance metrics in a table (see fig. 7)
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Fisher, Karkoub, Moselhi, and Thompson as applied to claim 11 above, and further in view of Taheri (US 20110144356 A1).
With respect to claim 14, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 11. Fisher further teaches wherein said shell and tube device is a reactor (“The discussion below, with reference to the steps provided at FIG. 15, the image provided at FIG. 5, and the user interface display illustrations of FIGS. 6A and 6B, provides an example of a method for tracking a robot on a surface such as a tube sheet of a nuclear reactor.” Paragraph 0054), but does not teach where it is utilized to perform a chemical conversion.
Taheri teaches conversion of propylene to acrolein and/or acrylic acid (“This invention also comprises a process for preparation of acrylic acid by oxidation of propylene using at least one oxidation reactor comprising a plurality of walled conduits according to the invention. Typically, the catalytic vapor phase oxidation of propylene is carried out in two stages, i.e., oxidation of propylene to acrolein and acrylic acid” paragraph 0085); conversion of acrolein to acrylic acid (“oxidation of acrolein to acrylic acid” paragraph 0085); conversion of n-butane to maleic anhydride (“One preferred process of the invention comprises oxidation of n-butane to form maleic anhydride” paragraph 0036);
Taheri is analogous art in the same field of endeavor. Taheri is directed towards reactor that convert chemicals (“This invention also comprises a process for preparation of acrylic acid by oxidation of propylene using at least one oxidation reactor comprising a plurality of walled conduits according to the invention. Typically, the catalytic vapor phase oxidation of propylene is carried out in two stages, i.e., oxidation of propylene to acrolein and acrylic acid” paragraph 0085). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Fisher, Karkoub, Moselhi, and Thompson with Taheri by utilizing Taheri’s teachings of chemical conversion in conjunction with the tube system of Fisher, with the expectation that doing so would lead to improved reactor usage (“The invention is improved flow reactors for exothermic chemical conversions using a fixed heterogeneous catalyst with means for control of the exotherm. Apparatus of this invention is for use in a process which includes conversions of organic compounds to value added products using a selective heterogeneous catalyst.” Paragraph 0020)
Claims 15, 18 and 23-24 is rejected under 35 U.S.C. 103 as being unpatentable over Fisher, Karkoub, Moselhi, and Thompson as applied to claim 11 above, and further in view of Dragan.
With respect to claim 15, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 11 but do not teach it further comprising the step of using one or more data records stored in the relational database to produce one or more of tables, graphs, spreadsheets, and color-coded summary graphics.
Dragan teaches converting relational databases into spreadsheets and tables (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1)
Dragan is analogous art reasonably pertinent to the problem of data storage and processing. Dragan is directed towards a methodology of converting relational databases into spreadsheets and tables (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Fisher, Karkoub, Moselhi, and Thompson by utilizing Dragan’s teachings of data conversion, with the expectation that doing so would lead to the ability to analyze data present in the relational database (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1).
With respect to claim 18, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 11. Fisher additionally teaches measuring one or more workspace parameters (“Processing circuitry 50 (FIG. 14) translates data detected in images acquired by the robot camera into tube sheet space, so that positions of tubes that are within each acquired image are identifiable in tube sheet space and stored in memory” paragraph 0053 and figure 15 and “The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image…” paragraph 0038), but does not teach recording workspace parameter measurements in the relational database, and optionally, presenting said workspace parameter measurements on a visual display.
Moselhi teaches recording workspace parameter measurements in the relational database (“To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” and “There are different database models, of which the relational model is the most commonly used in engineering applications” paragraph 0308).
Dragan teaches optionally, presenting said workspace parameter measurements on a visual display (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1).
Dragan is analogous art reasonably pertinent to the problem of data storage and processing. Dragan is directed towards a methodology of converting relational databases into spreadsheets and tables (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Fisher, Karkoub, Moselhi, and Thompson by utilizing Dragan’s teachings of data conversion, with the expectation that doing so would lead to the ability to analyze data present in the relational database (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1).
With respect to claim 23, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 19, but do not teach it further comprising the step of using one or more data records stored in the relational database to produce one or more of tables, graphs, spreadsheets, and color-coded summary graphics.
Dragan teaches converting relational databases into spreadsheets and tables (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1).
Dragan is analogous art reasonably pertinent to the problem of data storage and processing. Dragan is directed towards a methodology of converting relational databases into spreadsheets and tables (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1). A person of ordinary skill in the art before the effective filing date of the claimed invention would have found it obvious to combine the teachings of Fisher, Karkoub, Moselhi, and Thompson by utilizing Dragan’s teachings of data conversion, with the expectation that doing so would lead to the ability to analyze data present in the relational database (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1).
With respect to claim 24, Fisher, Karkoub, Moselhi, and Thompson teach the method of claim 23. Fisher additionally teaches measuring one or more workspace parameters (“Processing circuitry 50 (FIG. 14) translates data detected in images acquired by the robot camera into tube sheet space, so that positions of tubes that are within each acquired image are identifiable in tube sheet space and stored in memory” paragraph 0053 and figure 15 and “The heat exchanger characteristics may include tube locations, or identification of plugged tubes, stay tubes, or the like, e.g. as identified by the centers thereof. The processing circuitry compares the heat exchanger characteristics from the image…” paragraph 0038), but does not teach recording workspace parameter measurements in the relational database, and optionally, presenting said workspace parameter measurements on a visual display.
Moselhi teaches recording workspace parameter measurements in the relational database (“To assist municipality engineers in carrying out this challenging task, an automated rehabilitation system has been developed. The system consists of two main modules, a DBMS and a DSS, developed in Microsoft Access and Visual Basic environments, respectively. FIG. 92 depicts the main modules of the developed system. As depicted in FIG. 92, the selection process of a suitable rehabilitation technique starts by feeding the system with a report on the status of defects. If the report indicates no sign of defects, then the rehabilitation system will not be executed and a report will be issued accordingly. Otherwise the user will be required to input necessary information” and “There are different database models, of which the relational model is the most commonly used in engineering applications” paragraph 0308).
Dragan teaches optionally, presenting said workspace parameter measurements on a visual display (“Data loading into a SQL Server table and extracting data from a SQL Server table (ETL) is very important. Microsoft Excel is maybe one of the most popular programs on Windows for these tasks, due to the fact that a spreadsheet is similar to a SQL Server table and is very easy to input data in a table format, visualize data in a spreadsheet, as well as processing it for further needs.” Page 1).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Tripathy (US 20160025961 A1)- discloses a multi-tube inspection system
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REBECCA C WILLIAMS whose telephone number is (571)272-7074. The examiner can normally be reached M-F 7:30am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew W Bee can be reached at (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/REBECCA COLETTE WILLIAMS/Examiner, Art Unit 2677
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666