DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The claims will be read under the broadest reasonable interpretation standard outlined in
MPEP § 2111.01.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims of copending Application No. 18/891,485 (Hereinafter, “485”). Although the claims at issue are not identical, they are not patentably distinct from each other. Reference may be made to the limitation mappings/table presented below in view of representative claims of 485.
The instant claims and the claims of reference recite common subject matter, and recite the open-ended transitional phrase “comprising” which does not preclude any additional elements recited by claims of reference.
Language/terminology of the instant claims constituting minor/slight variations from the
claims of reference, if/where present, require interpretations under Broadest Reasonable
Interpretation and/or plain meaning definitions (MPEP 2173 and 2111) equivalent to/met by
language of the reference claims in view of that corresponding Specification. While the
disclosure of reference may not be used as prior art (Double Patenting concerns the claims of
reference), portions of the specification which provide support for reference claims may also be
examined and considered when addressing the scope of claims of reference and the issue of
whether an instant claim defines an obvious variation or falls within the scope of an invention
claimed in the claim(s) of reference. See MPEP 804 with reference to In re Vogel, 422 F.2d 438,
164 USPQ 619 (CCPA 1970).
Whereby elements of the instant claims otherwise not present explicitly in corresponding reference claims correspond to disclosure as identified in Jorud et. al (US 20210102454 A1) (Hereinafter, “Jorud”), it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify claims of reference such that the teachings may further include detection of motion with respect to a catwalk system in a wellsite. Doing so would have the advantage of greater monitoring worker safety in increased accuracy, by considering this precise component in a broader system of safety control.
It would have been further obvious to modify the claims of reference to include the usage of multiple detectors for detecting motion of both objects and humans, as taught by Jorud. Doing so would provide greater area of coverage and scope of recognition by the broader system.
It would have been further obvious to modify the claims of reference to include consideration of the predetermined zones of danger with respect to a catwalk, as taught by Jorud. Doing so would provide a clear barrier for the system to recognize as a threshold for implementing safety measures.
It would have been further obvious to modify the claims of reference to include the alarm system of Jorud. Doing so would have the advantage of providing increased safety to workers within the monitoring system.
The teachings of Jorud and the claims of reference readily integrate, as both are directed to an imagery-based system of monitoring in a wellsite setting. The claims of reference already consider the motion of a pipe, which could be readily expected to move via a pusher of a catwalk system. Jorud merely represents additional observations that could be included under the broader teaching without compromise to underlying functionality.
Whereby elements of the instant claims otherwise not present explicitly in corresponding reference claims correspond to disclosure as identified in Jorud and Magnuson et. al (US 20190186216 A1) (Hereinafter, “Magnuson”), it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the claims of reference such that the image analysis greater considers the motion of components of a catwalk. Doing so provides a level greater of specificity within the monitoring teachings, to better identify the precise moments of risk. The teachings of Jorud/Magnuson and the claims of reference readily integrate, as both are directed to an imagery-based system of monitoring in a wellsite setting. The claims of reference already consider the motion of a pipe, which could be readily expected to move via a pusher of a catwalk system. Jorud/Magnuson merely represents additional observations that could be included under the broader teaching without compromise to underlying functionality.
Whereby elements of the instant claims otherwise not present explicitly in corresponding reference claims correspond to disclosure as identified in Jorud and Harshbarger et. al (US 20180363421 A1) (Hereinafter, “Harshbarger”), it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the claims of reference such that a confined space is considered. Doing so provides for more precise mathematical calculations in performing the object detection. The teachings readily integrate, as the pipe within the wellsite of the claims of reference could readily be expected to be in such a position.
Whereby elements of the instant claims otherwise not present explicitly in corresponding reference claims correspond to disclosure as identified in Jorud and Thompson et. al (US 20200106992 A1) (Hereinafter, “Thompson”), it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify claims of reference such that velocity of objects may be detected. Doing so provides increased information within the broader object tracking, to assist in activating downstream safety measures. The teachings readily integrate, as velocity can be readily calculated in a system which already tracks motion.
This is a provisional nonstatutory double patenting rejection because the patentably
indistinct claims have not in fact been patented.
Claims of Reference (485)
Instant Claims (18/891,635)
Claim 1
A method comprising:
receiving imagery data from a wellsite - in further view of Jorud ([0033]; [0053]-[0054])
analyzing the imagery data to detect movement - in further view of Jorud (Fig. 1; Fig. 3; [0033]; [0053]-[0054]; [0064]; [0066]; [0069]; [0072]; [0086]; [0097])
determining a risk to a human at the wellsite based on the detected movement
responsive to the determining, issuing an instruction to reduce the risk
Claim 1
A method comprising:
receiving imagery data from a wellsite that comprises a catwalk system
analyzing the imagery data to detect movement with respect to the catwalk system
determining a risk to a human at the wellsite based on the detected movement
responsive to the determining, issuing an instruction to reduce the risk.
Claim 2
The method of claim 1, wherein the movement corresponds to a physical object
Claim 2
The method of claim 1, wherein the movement corresponds to a physical object
Claim 3
The method of claim 2, wherein the physical object comprises a pipe that comprises a pipe end – in further view of Jorud ([0021])
Claim 3
The method of claim 2, wherein the physical object comprises a pipe movable by a pusher of the catwalk system
Claim 4
The method of claim 3, wherein the determining comprises assessing movement of the pipe and the pipe end with respect to a position of the human
Claim 4
The method of claim 3, wherein the determining comprises assessing movement of the pipe with respect to a position of the human
Claim 1
in further view of Jorud and Magnuson (Fig. 2; [0016]; [0017])
Claim 5
The method of claim 1, wherein the analyzing comprises detecting a state of a pusher of the catwalk system and detecting a pipe positioned with respect to the pusher.
Claim 1
in further view of Jorud ([0021]) and Magnuson ([0014]); Magnuson ([0017])
Claim 6
The method of claim 5, wherein the state of the pusher comprises a position of the pusher within a confined space.
Claim 1
in further view of Jorud and Harshbarger (Fig. 1; [0020])
Claim 7
The method of claim 1, wherein the analyzing comprises determining a confined space of movement of a pusher of the catwalk system.
Claim 1
in further view of Jorud and Thompson ([0006])
Claim 8
The method of claim 1, wherein the analyzing comprises determining a velocity of a pusher of the catwalk system.
Claim 1
in further view of Jorud ([0053]-[0054])
Claim 9
The method of claim 1, comprising implementing a first detector for detecting the movement and a second detector for detecting presence of a human.
Claim 1
in further view of Jorud (Fig. 3; [0021]; [0053]-[0054]; [0072]-[0073])
Claim 10
The method of claim 9, wherein the second detector utilizes a predefined zone that is characterized by a distance or distances with respect to one or more components of the catwalk system, wherein the catwalk system comprises a pusher for moving pipe.
Claim 1
in further view of Jorud ([0073])
Claim 11
The method of claim 1, wherein the instruction comprises an instruction to energize a siren.
Claim 12
The method of claim 1, wherein one or more of the analyzing and the determining comprise implementing at least one model.
Claim 12
The method of claim 1, wherein one or more of the analyzing and the determining comprise implementing at least one model.
Claim 13
The method of claim 12, wherein the at least one model comprises one or more of a physics-based model and a machine learning-based model.
Claim 13
The method of claim 12, wherein the at least one model comprises one or more of a physics-based model and a machine learning-based model.
Claim 14
The method of claim 1, wherein the analyzing comprises utilizing a neural network model to detect an object.
Claim 14
The method of claim 1, wherein the analyzing comprises utilizing a neural network model to detect an object.
Claim 15
The method of claim 14, wherein the neural network model comprises a you-only-look-once (YOLO) model.
Claim 15
The method of claim 14, wherein the neural network model comprises a you-only-look-once (YOLO) model.
Claim 16
The method of claim 14, wherein the analyzing further comprises utilizing an optical flow process to detect movement of a portion of the object.
Claim 16
The method of claim 14, wherein the analyzing further comprises utilizing an optical flow process to detect movement of the object.
Claim 17
The method of claim 16, wherein the object is a pipe and wherein the portion is an end of the pipe.
Claim 17
The method of claim 16, wherein the object is a pipe.
Claim 18
The method of claim 17, wherein the instruction comprises an instruction to control position of the end of the pipe or an instruction for a human to move.
Claim 18
The method of claim 17, wherein the instruction comprises an instruction to control position of the pipe or an instruction for a human to move.
Claim 19
A system comprising
a processor
memory accessible by the processor
processor-executable instructions stored in the memory and executable to instruct the system to:
receive imagery data from a wellsite - in further view of Jorud ([0033]; [0053]-[0054])
analyze the imagery data to detect movement - in further view of Jorud (Fig. 1; Fig. 3; [0033]; [0053]-[0054]; [0064]; [0066]; [0069]; [0072]; [0086]; [0097])
make a determination as to a risk to a human at the wellsite based on the detected movement
and responsive to the determination, issuing an instruction to reduce the risk.
Claim 19
A system comprising:
a processor
memory accessible by the processor
processor-executable instructions stored in the memory and executable to instruct the system to:
receive imagery data from a wellsite that comprises a catwalk system
analyze the imagery data to detect movement with respect to the catwalk system
make a determination as to a risk to a human at the wellsite based on the detected movement
and responsive to the determination, issuing an instruction to reduce the risk
Claim 20
One or more computer-readable storage media comprising processor-executable instructions to instruct a computing system to: receive imagery data from a wellsite - in further view of Jorud ([0033]; [0053]-[0054])
analyze the imagery data to detect movement - in further view of Jorud (Fig. 1; Fig. 3; [0033]; [0053]-[0054]; [0064]; [0066]; [0069]; [0072]; [0086]; [0097])
make a determination as to a risk to a human at the wellsite based on the detected movement
and responsive to the determination, issuing an instruction to reduce the risk.
Claim 20
One or more computer-readable storage media comprising processor-executable instructions to instruct a computing system to: receive imagery data from a wellsite that comprises a catwalk system
analyze the imagery data to detect movement with respect to the catwalk system
make a determination as to a risk to a human at the wellsite based on the detected movement
and responsive to the determination, issuing an instruction to reduce the risk.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because they are directed to ineligible patent subject matter. The claims are directed to the Abstract Idea groupings of mathematical calculations under MPEP § 2106.04(a)(2)(I) and mental processes under MPEP § 2106.04(a)(2)(III). These are judicial exceptions under Step 2A, Prong One of the framework established by the cases of Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 573 U.S. 208, 216, 110 USPQ2d 1976, 1980 (2014) and Mayo Collaborative Servs. v. Prometheus Labs., Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 (2012). See MPEP § 2106.04(II).
PNG
media_image1.png
200
400
media_image1.png
Greyscale
Step 1: The claims in question are directed to a method, device, and non-transitory ([0079] of the claimed invention’s specification) computer-readable media (CRM) for a “wellsite operations machine vision framework.” Machines, processes, and articles of manufacture (non-transitory CRM) are all statutory categories. See MPEP 2106.03(I), “A machine is a "concrete thing, consisting of parts, or of certain devices and combination of devices." Digitech, 758 F.3d at 1348-49, 111 USPQ2d at 1719 (quoting Burr v. Duryee, 68 U.S. 531, 570, 17 L. Ed. 650, 657 (1863)). This category "includes every mechanical device or combination of mechanical powers and devices to perform some function and produce a certain effect or result." Nuijten, 500 F.3d at 1355, 84 USPQ2d at 1501 (quoting Corning v. Burden, 56 U.S. 252, 267, 14 L. Ed. 683, 690 (1854))”; See MPEP 2106.03(I), “NTP, Inc. v. Research in Motion, Ltd., 418 F.3d 1282, 1316, 75 USPQ2d 1763, 1791 (Fed. Cir. 2005) ("[A] process is a series of acts.") (quoting Minton v. Natl. Ass’n. of Securities Dealers, 336 F.3d 1373, 1378, 67 USPQ2d 1614, 1681 (Fed. Cir. 2003)). As defined in 35 U.S.C. 100(b), the term "process" is synonymous with "method."”; See MPEP 2106.03(I), “A manufacture is "a tangible article that is given a new form, quality, property, or combination through man-made or artificial means." Digitech, 758 F.3d at 1349, 111 USPQ2d at 1719-20 (citing Diamond v. Chakrabarty, 447 U.S. 303, 308, 206 USPQ 193, 197 (1980)). As the courts have explained, manufactures are articles that result from the process of manufacturing, i.e., they were produced "from raw or prepared materials by giving to these materials new forms, qualities, properties, or combinations, whether by hand-labor or by machinery." Samsung Electronics Co. v. Apple Inc., 137 S. Ct. 429, 120 USPQ2d 1749, 1752-3 (2016) (quoting Diamond v. Chakrabarty, 447 U. S. 303, 308, 206 USPQ 193, 196-97 (1980)); Nuijten, 500 F.3d at 1356-57, 84 USPQ2d at 1502.”; See MPEP 2016.03(II) (distinguishing statutory non-transitory memory and non-statutory carrier waves, as relevant to [0079]-[0080] of the claimed invention’s specification). (Step 1: Yes).
Step 2A, Prong One: As explained in MPEP 2106.04(II), a claim “recites” a judicial
exception when the judicial exception is “set forth” or “described” in the claim. Here, each
claim recites or depends upon the mental processes of analyzing imagery data, detecting movement, determining risk, and issuing instruction(s). (Claim 1 “analyzing the imagery…determining a risk…responsive to the determining, issuing an instruction to reduce the risk…”; Claim 4 “determining comprises assessing movement of the pipe with respect to a position of the human”; Claim 5 “analyzing comprises detecting a state of a pusher of the catwalk system and detecting a pipe positioned with respect to the pusher”; Claim 7 “analyzing comprises determining a confined space of movement of a pusher of the catwalk system”; Claim 8 “analyzing comprises determining a velocity of a pusher of the catwalk system”; Claim 11 “wherein the instruction comprises an instruction to energize a siren”)
The claims are recited at a high level of generality and lack any specifics precluding such
an analysis from being interpreted under the mental processes grouping of “practically performed
in the mind” (see also MPEP § 2106.04(a)(2) identifying how e.g. a use of pen and paper, a ruler,
or a computer as a tool (to assist in visually/mentally analyzing/observing acquired
images/video) fails to preclude such an interpretation under the mental processes judicial
exception). Activities such as “receiving imagery data” therefore may be performed mentally, even if they may require the additional computer tool.
Regarding artificial intelligence, to the extent it is implicated, claims 12-17 are comparable to Claim 2 of Example 47 of the July 2024 PEG regarding subject matter eligibility (https://www.uspto.gov/sites/default/files/documents/2024-AISMEUpdateExamples47-49.pdf). As stated therein, an artificial intelligence’s analyses, detections, and reinforcement learnings may be practically performed in the human mind. To the extent mathematical calculations are required to operate and train the artificial intelligence in image analysis, the separate judicial exception is also implicated.
As such, the usage of a computer to detect movement, determine risk, and generate instruction to reduce risk does not elevate these claims beyond a mental process. (Step 2A, Prong One: Yes).
Step 2A, Prong Two: If Prong One of Step 2A is met, the examiner must consider (1)
whether there are any ‘additional elements’ recited in the claim beyond the judicial exception,
and (2) evaluate those additional elements individually and in combination to determine whether
the claim as a whole integrates the exception into a practical application. See MPEP §
2106.04(d).
Limitations the courts have found indicative of integration include: an improvement in
the functioning of a computer, or an improvement to other technology or technical field, as
discussed in MPEP §§ 2106.04(d)(1) and 2106.05(a); applying or using a judicial exception to
effect a particular treatment or prophylaxis for a disease or medical condition, as discussed in
MPEP § 2106.04(d)(2); implementing a judicial exception with, or using a judicial exception in
conjunction with, a particular machine or manufacture that is integral to the claim, as discussed
in MPEP § 2106.05(b); effecting a transformation or reduction of a particular article to a
different state or thing, as discussed in MPEP § 2106.05(c); and applying or using the judicial
exception in some other meaningful way beyond generally linking the use of the judicial
exception to a particular technological environment, such that the claim as a whole is more than
a drafting effort designed to monopolize the exception, as discussed in MPEP § 2106.05(e).
Limitations that the courts have found non-indicative of integration include: merely
reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including
instructions to implement an abstract idea on a computer, or merely using a computer as a tool to
perform an abstract idea, as discussed in MPEP § 2106.05(f); adding insignificant extra-solution
activity to the judicial exception, as discussed in MPEP § 2106.05(g); and generally linking the
use of a judicial exception to a particular technological environment or field of use, as discussed
in MPEP § 2106.05(h).
As an additional note, ‘additional elements’ are generally limitations excluded from
interpretation under the Abstract Idea groupings, and may comprise portions of limitations
otherwise identified as falling under those Abstract Idea groupings of the 2019 PEG (e.g. any
‘determination’ that may be made mentally by a user, neural network and/or generic computer hardware is considered under the ‘apply it’ considerations of 2106.05(f)). Any ‘providing’/outputting broadly, and ‘collection/input’ of data (i.e. receiving imagery data, issuing an instruction), also fail(s) to integrate at least in view of MPEP 2106.05(g) (extra-solution data gathering/output) and/or 2106.05(h) as ‘generally linking’ the exception to a field of use involving machine learning and/or imagery so acquired (e.g. the use of a computer to acquire said image broadly). The same determination holds for dependent claims that serve to limit the collection/output of data/images (by means of what is collected based on recited conditions) and/or introduce limitations generally linking to a field of use.
None of the instant claims appear to explicitly/clearly capture/recite any disclosed
improvement in technology (see MPEP 2106.05(a), with note that ‘functioning of a computer’
concerns functions integral to the way a computer operates and not ‘functions’ that a generic
computer can be programmed/adapted to perform (see also 2106.05(f))) and any ‘additional
elements’, even when considered in combination, fail to integrate at Prong Two of Step 2A
accordingly. Integration in view of subsection (a) requires an identification of the manner in
which the improvement is achieved, to be explicitly and specifically recited in the claims, as
‘additional elements’ precluded from interpretation under any of the Abstract Idea groupings
(since the improvement cannot be to the exception itself). With reference to MPEP 2106.05(a):
It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. See the discussion of Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981))
As applicable here, additional limitations not directed to a judicial exception fail to
integrate at Prong Two of Step 2A. Claim 1 recites a “a catwalk system”; Claim 3 recites a “a pipe movable by a pusher of the catwalk system”; Claim 12 recites “one model”; Claim 15 recites “a neural network model”; Claim 16 recites a “you-only-look-once (YOLO) model”; Claim 19 recites “a processor…memory accessible by the processor…”; Claim 20 recites “one or more computer-readable storage media”. The incorporation of conventional wellsite and computer/machine-learning systems does little more than generally link the judicial exceptions of mental processes and mathematical calculation to a field-of-use and technological environment. See MPEP §§ 2106.05(h); 2106.05(f).
Claim 1 recites “receiving imagery data”, and “issuing an instruction to reduce the risk”. These limitations constitute insignificant extra-solution activity under MPEP § 2106.05(g). Specifically, the limitations amount to no more than necessary data inputting and outputting, under rationale 3 of MPEP § 2106.05(g).
Even when viewed in combination, any additional elements present do not integrate the
recited judicial exception into a practical application (Step 2A, Prong Two: No), and the claims
are directed to the judicial exception. (Revised Step 2A: Yes → Step 2B).
Step 2B: If Prong Two of Step 2A is not met, the examiner must consider whether the
claim as a whole amounts to ‘significantly more’ than the recited exception, i.e., whether any
‘additional element’, or combination of additional elements, adds an inventive concept to the
claim. The considerations of Step 2A Prong 2 and Step 2B overlap, but differ in that 2B also
requires considering whether the claims feature any “specific limitation(s) other than what is
well-understood, routine, conventional activity in the field” (WURC) (MPEP § 2106.05(d)).
Such a limitation if specifically recited however, must still be excluded from interpretation under
any of the Abstract Idea groupings. Step 2B further requires a re-evaluation of any additional
elements drawn to extra-solution activity in Step 2A (e.g. receiving imagery data, issuing an instruction) – however no limitations appear directed to any novel collection or output generation per se. Limitations not indicative of an inventive concept/‘significantly more’ include those that are not specifically recited (instead recited at a high level of generality), those that are established as WURC (a plurality of cited references serve to evidence the WURC nature of ‘analysis’ based at least in part on corroborating/additional ground data), and/or those that are not ‘additional elements’ by nature of their analysis at Prong One of Step 2A (i.e. directed to the exception – see above re. deciding that a second acquisition may be advantageous/desired). The July 2024 PEG describes that an improvement/ inventive concept (for ‘significantly more’ determination(s)) cannot be to the judicial exception itself. As additionally recited by the specification of the claimed invention, machine learning is understood to encompass a plurality of WURC machine-learning models, including [00124] the YOLO model(s) known in the art, as well as [00128] R-CNNs and CNNs described broadly.
The claims in question recite little beyond those limitations recited at a high level
of generality and falling under e.g. the mental processes and mathematical calculation Abstract Idea groupings, and would monopolize the exceptions accordingly. The additional limitations of computer processing, machine-learning, and input/output as recited are WURC, as evidenced by the body of prior art cited by the examiner in this office action. (Step 2B: No).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 9-12, and 19-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jorud et. al (US 20210102454 A1) (Hereinafter, “Jorud”).
With respect to claim 1, Jorud discloses:
A method ([Abstract]) comprising:
receiving imagery data from a wellsite ([0033]; [0053]-[0054]) that comprises a catwalk system ([0033] “For example, the video cameras 198 may capture videos of the entire well construction system 100 and/or specific portions of the well construction system 100, such as the drill floor 114, the top drive 116, the iron roughneck 165, the THM 160, the fingerboard 164, and/or the catwalk 161, among other examples.”)
analyzing the imagery data to detect movement with respect to the catwalk system (Fig. 1; Fig. 3; [0033]; [0053]-[0054]; [0064]; [0066] “The drill floor equipment 301, such as…the catwalk 161”; [0069]; [0072]; [0086]; [0097])
PNG
media_image2.png
117
321
media_image2.png
Greyscale
PNG
media_image3.png
561
320
media_image3.png
Greyscale
determining a risk to a human at the wellsite based on the detected movement ([0053]-[0054] “Thus, if a wellsite operator 195 walks into the restricted area 306, the processing device 192 and/or the local controllers may then cause the drill floor equipment 301 located within the restricted area 304 of the drill floor 114 to enter or change to the safe mode of operation”; [0072] “These ramp-down procedures can be circumvented if the presence detection sensors 198, 342, 350 detect that a wellsite operator 195 has crossed a barrier 312”; [0073]-[0074] further interpreting pressing of the button after initiation of safety protocols as a further determination of risk, “based on the detected movement” (since motion detection begins the process to begin with))
responsive to the determining, issuing an instruction to reduce the risk ([0053]-[0054]; [0072]-[0074])
With respect to claim 2, Jorud discloses:
The method of claim 1, wherein the movement corresponds to a physical object ([0033] “For example, the video cameras 198 may capture videos of the entire well construction system 100 and/or specific portions of the well construction system 100, such as the drill floor 114, the top drive 116, the iron roughneck 165, the THM 160, the fingerboard 164, and/or the catwalk 161, among other examples”; [0054] “The presence detection sensors 342 may also or instead be or comprise one or more laser imaging, detection and ranging (LIDAR) sensors…The processing device 192 may be operable to receive and process the information output by the LIDAR sensors 350 and, based on the received information, determine position and movement of selected drill floor equipment 301 and of the wellsite operators 195, and change operation (e.g., stop or slow down) of selected drill floor equipment 301 to avoid collision with other drill floor equipment 301, structures (e.g., support structure 112), and/or the wellsite operators 195”)
With respect to claim 3, Jorud discloses:
The method of claim 2, wherein the physical object comprises a pipe movable by a pusher of the catwalk system (Fig. 1, noting drill string 120 on drill floor 114; Fig. 3; [0021] “The catwalk 161 may comprise a skate 163 movable along a groove (not shown) extending longitudinally along the horizontal and inclined portions of the catwalk 161. The skate 163 may be operable to convey (e.g., push) the tubulars along the catwalk 161 to the drill floor 114”; [0054] “The LIDAR sensors 350 may facilitate detection and determination of geometry and position of physical objects on the drill floor 114, such as the wellsite operators 195, the drill floor equipment 301, and unexpected objects, such as tools, drill bits, slips 161, among other examples.”; [0086] “The drill floor equipment 301 may be or comprise at least one of drill string slips 161, a draw works 118, 218, an iron roughneck 165, a catwalk 161, a top drive 116, a TDA 202, and an LSA 228.”)
With respect to claim 4, Jorud discloses:
The method of claim 3, wherein the determining comprises assessing movement of the pipe with respect to a position of the human ([0054] “based on the received information, determine position and movement of selected drill floor equipment 301 and of the wellsite operators 195, and change operation (e.g., stop or slow down) of selected drill floor equipment 301 to avoid collision with…the wellsite operators 195”)
With respect to claim 9, Jorud discloses:
The method of claim 1, comprising implementing a first detector for detecting the movement ([0053]-[0054] “The processing device 192 may be operable to receive and process the information output by the LIDAR sensors 350 and, based on the received information, determine position and movement of selected drill floor equipment 301 and of the wellsite operators 195, and change operation (e.g., stop or slow down) of selected drill floor equipment 301 to avoid collision with other drill floor equipment 301, structures (e.g., support structure 112), and/or the wellsite operators 195.”) and a second detector for detecting presence of a human ([0053]-[0054] noting “one or more” language)
With respect to claim 10, Jorud discloses:
The method of claim 9, wherein the second detector utilizes a predefined zone that is characterized by a distance or distances with respect to one or more components of the catwalk system (Fig. 3; [0053]-[0054] “The video cameras 198 and the processing device 192 may utilize object recognition software to recognize wellsite operators 195 entering or present in the restricted area 306. Thus, if a wellsite operator 195 walks into the restricted area 306, the processing device 192 and/or the local controllers may then cause the drill floor equipment 301 located within the restricted area 304 of the drill floor 114 to enter or change to the safe mode of operation”; [0072]-[0073] “These ramp-down procedures can be circumvented if the presence detection sensors 198, 342, 350 detect that a wellsite operator 195 has crossed a barrier 312”), wherein the catwalk system comprises a pusher for moving pipe ([0021])
With respect to claim 11, Jorud discloses:
The method of claim 1, wherein the instruction comprises an instruction to energize a siren ([0073] “Red warning lights 364 for the activated and operating drill for equipment 301 may be initiated, pre-warning audio alarms can be sounded, and the selected drill floor equipment 301 can be started and used either fully or in a limited mode”)
With respect to claim 12, Jorud discloses:
The method of claim 1, wherein one or more of the analyzing and the determining comprise implementing at least one model ([0082] “The program code instructions 432 may further comprise modeling or predictive routines, equations, algorithms, processes, applications, and/or other programs operable to perform example methods and/or operations described herein”)
With respect to claim 19, Jorud discloses:
A system ([Abstract]) comprising:
a processor ([0086])
memory accessible by the processor ([0077])
processor-executable instructions stored in the memory and executable to instruct the system to ([0077])
receive imagery data from a wellsite that comprises a catwalk system ([0033]; [0053]-[0054])
analyze the imagery data to detect movement with respect to the catwalk system (Fig. 1; Fig. 3; [0033]; [0053]-[0054]; [0064]; [0066]; [0069]; [0072]; [0086]; [0097])
make a determination as to a risk to a human at the wellsite based on its detected movement ([0053]-[0054]; [0072]-[0074])
responsive to the determination, issuing an instruction to reduce the risk ([0053]-[0054]; [0072]-[0074])
With respect to claim 20, Jorud discloses:
One or more computer-readable storage media comprising processor-executable instructions to instruct a computing system ([0077]; [0086]) to:
receive imagery data from a wellsite that comprises a catwalk system ([0033]; [0053]-[0054])
analyze the imagery to detect movement with respect to the catwalk system (Fig. 1; Fig. 3; [0033]; [0053]-[0054]; [0064]; [0066]; [0069]; [0072]; [0086]; [0097])
make a determination as to a risk to a human at the wellsite based on the detected movement ([0053]-[0054]; [0072]-[0074])
responsive to the determination, issuing an instruction to reduce the risk ([0053]-[0054]; [0072]-[0074])
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Jorud in view of Magnuson et. al (US 20190186216 A1) (Hereinafter, “Magnuson”)
With respect to claim 5, Jorud teaches the method of claim 1.
Jorud does not explicitly teach:
wherein the analyzing comprises detecting a state of a pusher of the catwalk system and detecting a pipe positioned with respect to the pusher
However, Magnuson, in the same field of endeavor of worksite monitoring, teaches:
wherein the analyzing comprises detecting a state of a pusher of the catwalk system and detecting a pipe positioned with respect to the pusher (Fig. 2; [0016] detecting pipe; [0017] detecting skate (pusher) and pipe position with respect to pusher)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Jorud to include the limitations of pusher and pipe detection, as taught by Magnuson. Doing so would have the advantage of detecting more specifically these exact components, for any additional dangers they would impose on operators. The systems readily integrate, as Jorud is already configured to monitor a wide variety of equipment on the drill floor (Fig. 1; Fig. 3; [0054] “The LIDAR sensors 350 may facilitate detection and determination of geometry and position of physical objects on the drill floor 114, such as the wellsite operators 195, the drill floor equipment 301, and unexpected objects, such as tools, drill bits, slips 161, among other examples”). Magnuson further integrates, as it does not rule out additional types of “proximity sensors” ([0017] “Proximity sensor 111 may be any sensor known in the art, including, for example and without limitation, one or more of an ultrasonic sensor, photoelectrical sensor, or inductive sensor”)
With respect to claim 6, Jorud and Magnuson teach the method of claim 5.
Jorud and Magnuson further teach:
wherein the state of the pusher comprises a position of the pusher within a confined space (Jorud [0021] “The catwalk 161 may comprise a skate 163 movable along a groove (not shown) extending longitudinally along the horizontal and inclined portions of the catwalk 161”; Magnuson [0014] “Skate 109 may be a structure adapted to move along the length of carrier 107”; Magnuson [0017] “In some embodiments, the measured position of skate 109 along carrier 107 may be used to determine the distance between sensitive axis or plane 113 of proximity sensor 111 and skate 109.”; Read in line with [00122] of the claimed invention’s specification, which states “As an example, such a detector may utilize one or more techniques applied to digital imagery, which may include processing as to edges and/or objects and/or, for example, identification of equidistant points along a previously known line of movement to which a pusher may be confined. For example, consider a registration approach that may register movements (e.g., movement limits, etc.) of a pusher such that a point-based approach may be utilized given a confined movement space”)
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Jorud in view of Harshbarger et. al (US 20180363421 A1) (Hereinafter, “Harshbarger”)
With respect to claim 7, Jorud teaches the method of claim 1.
Jorud does not explicitly teach:
wherein the analyzing comprises determining a confined space of movement of a pusher of the catwalk system
However, Harshbarger, in the same field of endeavor of worksite monitoring, teaches:
wherein the analyzing comprises determining a confined space of movement of a pusher of the catwalk system (Fig. 1, showing confined space of movement being determined; [0020] “In one example, the operating sensors 155 may be used to determine the number of times the trough 115 is raised and lowered by measuring the number of strokes of the piston/cylinders 160 and/or the number of cycles that the piston/cylinders 160 have been pressurized. In another example, the operating sensors 155 may be used to determine the weight of the tubular 106 by measuring the load on the trough 115 and/or the cross bars 161. In another example, the operating sensors 155 may be used to determine the number of cycles that the skate 130 has been actuated by measuring the proximity of the skate 130 to each end of the trough 115”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Jorud to include the limitations of confined space movement determination, as taught by Harshbarger. Doing so has the advantage of tracking the movements of the pusher specifically using known limits of its confinement with respect to the trough. The systems readily integrate, as the image analysis of Jorud may substitute for the operating sensors of Harshbarger to predictable success, since conventional image analysis is also capable of detecting motion.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Jorud in view of Thompson et. al (US 20200106992 A1) (Hereinafter, “Thompson”)
With respect to claim 8, Jorud teaches the method of claim 1.
Jorud does not explicitly teach:
wherein the analyzing comprises
(As Jorud tracks position if not velocity explicitly, of various drill floor equipment including the catwalk which the pusher is housed within [0021]; [0053]-[0054])
However, Thompson, reasonably pertinent to the problem of object tracking, teaches:
determining a velocity [from an image] ([0006] “In an embodiment, the equipment object is a first equipment object, and recognizing the event includes: determining and comparing steps. Determining includes determining, from the image data, a parameter of a first data set, the parameter being: a size of a second equipment object, a shape of the second equipment object, a position of the second equipment object, a physical condition of the second equipment object, a trajectory of the second equipment object, a travel path of the second equipment object, a velocity of the second equipment object, an acceleration or a deceleration of the second equipment object, a distance of the second equipment object from the first equipment object, or a registration marker of the second equipment object. Comparing includes comparing the parameter of the first data set with a corresponding second parameter of a second data set, wherein the corresponding second parameter is retrieved from a reference source.”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Jorud to include the limitations of detecting velocity of objects from image data. Doing so would allow the system to predict trajectories and operate with greater responsiveness to safety risks. The systems readily integrate, as Jorud is already configured to track the position of the pusher ([0024]; [0053]-[0054])
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Jorud in view of Wei et. al Safety assessment of existing pin-jointed grid structures with crooked members using static model updating (Hereinafter, “Wei”) and Li et. al Study on the Interaction Behaviors Identification of Construction Workers Based on ST-GCN and YOLO (Hereinafter, “Li”)
With respect to claim 13, Jorud teaches the method of claim 12
Jorud does not explicitly teach:
wherein the at least one model comprises one or more of a physics-based model and a machine learning-based model
However, Wei, in the same field of endeavor of safety monitoring, teaches:
wherein the at least one model comprises one or more of a physics-based model (Fig. 6; [5.2] “The bending deformation of the crooked members is measured by 3D laser scanning and image-based 3D reconstruction, wherein the former can provide more precise measuring data. Here, the CREAFORM MetraScan750TM scanning system with an average deviation of 0.03 mm is employed, as shown in Fig. 7. In this study, the measuring data from the 3D laser scanning are utilized in the model updating”).
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Jorud to include the limitations of physics-based modeling, as taught by Wei. Doing so would provide an additional means to determine structural risk to human operators. The systems readily integrate, as Jorud is already configured to include such modeling (Jorud [0082]) and further collect data that can be employed towards the same purpose.
Jorud and Wei do not explicitly teach:
and a machine learning-based model
However, Li, in the same field of endeavor of workplace safety monitoring, teaches:
and a machine learning-based model ([3.2] “Unsafe Behaviors Identification Based on YOLO and ST-GCN”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Jorud and Wei to include the limitations of machine-learning based modeling, as taught by Li. Doing so would provide an additional means to identify risk to workers based on their behavior. The systems readily integrate, as Jorud and Wei are already configured to include such modeling (Jorud [0082]) and further collect data that can be employed towards the same purpose.
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Jorud in view of Li
With respect to claim 14, Jorud teaches the method of claim 1
Jorud does not explicitly teach:
wherein the analyzing comprises utilizing a neural network model to detect an object
However, Li teaches:
wherein the analyzing comprises utilizing a neural network model to detect an object ([3.2.3] “Objects Detection Technology”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Jorud to include the limitations of neural network object detection, as taught by Li. Doing so would provide an additional means to identify risk to workers. The systems readily integrate, as Jorud is already configured to include such modeling (Jorud [0082]) and further collect data that can be employed towards the same purpose.
With respect to claim 15, Jorud and Li teach the method of claim 14.
Jorud and Li further teach:
wherein the neural network model comprises a you-only-look-once (YOLO) model ([3.2.3] “In this study, YOLOv5 was adopted for objects detection, which is an advanced object detection algorithm with important improvement in accuracy and speed compared to the previous YOLO versions”)
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Jorud and Li in view of Bhattacharjee et. al Two-Stream Convolutional Network with Multi-level Feature Fusion for Categorization of Human Action from Videos (Hereinafter, “Bhattacharjee”)
With respect to claim 16, Jorud and Li teach the method of claim 14
Jorud and Li do not explicitly teach:
wherein the analyzing further comprises utilizing an optical flow process to detect movement of the object
However, Bhattacharjee, in the same field of endeavor of object tracking, teaches:
wherein the analyzing further comprises utilizing an optical flow process to detect movement of the object ([2])
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Jorud and Li to include the limitations of optical flow object detection as taught by Bhattacharjee.
Claims 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Jorud, Li, and Bhattacharjee in view of Gustavsson et. al (US 20140233804 A1) (Hereinafter, “Gustavsson”)
With respect to claim 17, Jorud, Li and Bhattacharjee teach the method of claim 16.
Jorud, Li and Bhattacharjee do not explicitly teach:
wherein the object is a pipe
However, Gustavsson, in the same field of endeavor drill floor monitoring, teaches:
wherein the object is a pipe ([Abstract] “processing the data sets and/or images to identify the pipe in the data sets and/or images”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Jorud, Li and Bhattacharjee to include the limitations of pipe detection. Doing so would have the advantage of recognizing this specific component for the advantages of downstream safety control. The systems readily integrate, as Jorud is already configured to monitor many broader systems which a pipe is included within (Jorud [0094]).
With respect to claim 18, Jorud, Li, Bhattacharjee and Gustavsson teach the method of claim 17.
Jorud, Li, Bhattacharjee and Gustavsson further teach:
wherein the instruction comprises an instruction to control position of the pipe (Jorud [0072] “In some implementations triggering or operating a presence detection sensor 342 (e.g., crossing a light curtain, pulling a rope of a rope pull switch, stepping on a pressure mat, etc.), crossing the barriers 312, or triggering other presence detection sensors 198, 350 may cause the drill floor equipment 301 to stop immediately with no ramp-down period”) or an instruction for a human to move.
Additional References
Additionally cited references (see attached PTO-892) otherwise not relied upon above have been made of record in view of the manner in which they evidence the general state of the art.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NOAH WILLIAM BOYAR whose telephone number is (571)272-8392. The examiner can normally be reached 8:30 – 5:00 EST, Monday – Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NOAH W BOYAR/Examiner, Art Unit 2669
/CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights