Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office Action is in response to the application filed on 08/19/2024. Claim(s) 1-11 are pending in this application. Claim(s) 1,10 and 11 are independent claims.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Interpretation
“Consistent with the well-established axiom in patent law that a patentee or applicant is free to be his or her own lexicographer, a patentee or applicant may use terms in a manner contrary to or inconsistent with one or more of their ordinary meanings if the written description clearly redefines the terms”.- See MPEP 2173.05(a) .
“roll” in claims 2,6, and 9, per the instant specification at [0018] and [0044] , is interpreted as a parameter value passed between components of a workflow that define a semantic role/characteristic/classification of associated data. [0018] “The input roll column is an item that stores roll information, which defines a role of data input into model data. For example, in a case of character string data, a roll includes full name, sex (M, F, etc.), another name, nickname, trade name, product ID, or the like. In a case of numerical data, the roll includes age, category, or the like. In a case of image data, the roll stores a type of an object being an imaging target included in the image data such as document, identification card, driver's license, health insurance card, road, tire, and or human face.”
“screen” in claims 5 and 6, per the instant specification at [0022] and [0073] , is interpreted as a component encompassing an interface such as an application or software which when connected to other components in the generated information processing process may be used indicate a data source of input data to a subsequent connected component or data destination of output data to display data from a precedingly connected component. [0022] “The screen type column is an item that stores a type of a screen. The screen type includes input screen, notification screen, and the like. The input screen is a screen on which a user inputs input data for model data or function data. The notification screen is a screen on which a user is provided with notification of output data of the input screen, the model data, and the function data.” [0073] “FIG. 13 is a screen example illustrating a setting of a notification screen in the workflow creation process. … the user can select, as notification content, one or more pieces of output data to be the notification targets … that are input into the notification screen. The user can select, as the notification means, mail, a chat tool such as Slack, SMS, or the like by operating the input device 206 or the like of the user terminal 20.”
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention recites a judicial exception, is directed to that judicial exception, an abstract idea, it has not been integrated into practical application and the claims further do not recite significantly more than the judicial exception. Examiner has evaluated the claims under the framework provided in the 2019 Patent Eligibility Guidance published in the Federal Register 01/07/2019 and has provided such analysis below.
Regarding claim 1, the limitations, “accept a plurality of learned models selected by a user” and “generate an information processing process by combining the plurality of learned models accepted” as drafted, are functions that, under their broadest reasonable interpretation, recite the abstract idea of a mental process. These limitations encompass a human mind carrying out these functions through observation, evaluation judgment and /or opinion, or even with the aid of pen and paper. For example, the “accept” limitation can be carried out by a user approving or denying presented information. The “generate” limitation can be carried out by a user evaluating and combining accepted information with a pen and paper. Thus, these limitations recite and fall within the “Mental Processes” grouping of abstract ideas under Prong 1.
Claim 1 recites further additional elements, “An information processing apparatus comprising: processing circuitry configured to” and “learned models”. These additional elements are recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f). Therefore, the additional elements recited in claim 1 do not integrate the judicial exception into a practical application under prong 2, nor amount to significantly more under step 2B.
Regarding claim 2, the limitations, “generates the information processing process including a combination in which output data of a first learned model included in the plurality of learned models serves as input data of a second learned model included in the plurality of learned models,” and “an output data constraint including an output data type and an output roll of the first learned model is included in an input data constraint including an input data type and an input roll of the second learned model”, recites additional mental process under Prong 1. For example, the above limitations can be achieved by a user arranging and making determinations with various pieces of information with the aid of a pen and paper. The additional elements, “wherein the processing circuitry“, and “ learned model” and “second learned model”, are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f).
Regarding claim 3, the limitations, “generates the information processing process including a combination in which the output data of the first learned model included in the plurality of learned models serves as input data of a third learned model included in the plurality of learned models” and “the first learned model selectively outputs the output data to any one of the second learned model and the third learned model such that the output data constraint of the output data output from the first learned model is included in an input data constraint of the second learned model or the third learned model”, recites additional mental process under Prong 1. For example, the above limitations can be achieved by a user arranging and making determinations with various pieces of information with the aid of a pen and paper. The additional elements, “wherein the processing circuitry“, and “first/second/third learned model”, are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f).
Regarding claim 4, the limitations, “the first learned model: outputs the output data to the second learned model when the output data constraint of the output data output from the first learned model is included in the input data constraint of the second learned model; does not output the output data to the second learned model when the output data constraint of the output data output from the first learned model is not included in the input data constraint of the second learned model; outputs the output data to the third learned model when the output data constraint of the output data output from the first learned model is included in the input data constraint of the third learned model; and does not output the output data to the third learned model when the output data constraint of the output data output from the first learned model is not included in the input data constraint of the third learned model”, recites additional mental process under Prong 1. For example, the above limitations can be achieved by a user arranging and making determinations with various pieces of information with the aid of a pen and paper. The additional elements, “first/second/third learned model”, are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f).
Regarding claim 5, the limitations, “accepts a plurality of screens selected by the user” and “generates the information processing process by combining the plurality of learned models accepted and the plurality of screens accepted ”, recites additional mental process under Prong 1. For example, the above limitations can be achieved by a user arranging and making determinations with various pieces of information with the aid of a pen and paper. The additional elements, “processing circuitry”, “screens”, and “learned models” are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f).
Regarding claim 6, the limitations, “generates the information processing process including a combination in which output data of a first screen included in the plurality of screens serves as input data of the first learned model included in the plurality of learned models” and “specifies, based on an input data constraint including an input data type and input roll of the first learned model, an input field included in a first input screen and a data constraint including a data type and roll of the input field”, recites additional mental process under Prong 1. For example, the above limitations can be achieved by a user arranging and making determinations with various pieces of information with the aid of a pen and paper. The additional elements, “processing circuitry”, “screen”, and “learned model” are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f).
Regarding claim 7, the limitations, “accepts a plurality of functions selected by the user” and “generates a workflow by combining the plurality of functions accepted”, recites additional mental process under Prong 1. For example, the above limitations can be achieved by a user arranging and making determinations with various pieces of information with the aid of a pen and paper. The additional element, “processing circuitry”, and “learned models” are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f).
Regarding claim 8, the limitations,” generates the information processing process including a combination in which output data of the first learned model included in the plurality of learned models serves as input data of a first function included in the plurality of functions, and an output data type of the first learned model is included in an input data type of the first function”, recites additional mental process under Prong 1. For example, the above limitations can be achieved by a user arranging and making determinations with various pieces of information with the aid of a pen and paper. The additional elements, “processing circuitry”, are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f).
Regarding claim 9, the limitations, “presents, to the user, a plurality of functions that are selectable by the user, based on input data constraints including input data types and input rolls of the plurality of learned models accepted or output data constraints including output data types and output rolls of the plurality of learned models” and “accept a plurality of functions selected by the user from among the plurality of functions selectable by the user that are presented ”, recites additional mental process under Prong 1. For example, the above limitations can be achieved by a user arranging and making determinations with various pieces of information with the aid of a pen and paper. The additional elements, “processing circuitry”, are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f).
Regarding claim 10, the limitations,” accepting a plurality of learned models selected by a user;” and “generating an information processing process by combining the plurality of learned models accepted”, recites additional mental process under Prong 1. For example, the above limitations can be achieved by a user arranging and making determinations with various pieces of information with the aid of a pen and paper. The additional elements, “A non-transitory computer-readable storage medium, storing computer-readable instruction thereon, which, when executed by processing circuitry, cause the processing circuitry to execute”, are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f).
Regarding claim 11, the limitations, ”accepting a plurality of learned models selected by a user” and “generating an information processing process by combining the plurality of learned models accepted”, recites additional mental process under Prong 1. For example, the above limitations can be achieved by a user arranging and making determinations with various pieces of information with the aid of a pen and paper. The additional elements, “processing circuitry”, are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using generic computer, and/or generic computer components. See MPEP 2106.05(f).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 6 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6, recites the limitation " the plurality of screens" in the fifth line of the claim. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim 1,7 and 8, 10 and 11 are rejected under 35 U.S.C. 102 (a) (2) as being clearly anticipated by Gupte et al. (US 20220391176 A1) hereafter Gupte.
Regarding claim 1, Gupte teaches: An information processing apparatus comprising: processing circuitry configured to: ([0003] “Embodiments of the present disclosure relate to applications and platforms for configuring machine learning models for training and deployment using graphical components in a development environment. … The one or more processing operations may include operations that may be performed with respect to the model, data input to the model, and/or data output by the model such that the data and/or model may be configured in a certain manner that allows for deployment of the model according to certain parameters.”[0038] ”Various functions described herein as being performed by entities may be carried out by hardware, firmware, and/or software. For instance, various functions may be carried out by a processor executing instructions stored in memory.”)
The processor implemented method for carrying out one or more processing operations corresponds to, an information processing apparatus comprising: processing circuitry configured to.
accept a plurality of learned models selected by a user; and generate an information processing process by combining the plurality of learned models accepted.
([0018] “Combining components to perform a task may include linking graphical component objects in a graphical application editor such that an application may be generated by the graphical application editor by arranging the corresponding components and their respective underlying code based on the arrangement of the graphical component objects. The linked graphical component objects may be illustrative of the linking between components (e.g., the inputs and outputs of the corresponding components) as a component pipeline.” [0019] “Some components that may be visually represented in the graphical application editor may be configured as a machine learning model, or simply “model” in the present disclosure, unless otherwise defined.” [0023] “the component library may be displayed in the graphical application editor such that a user may be able to select one or more graphical component objects from the component library and include the selected graphical component objects in a canvas portion of the graphical application editor. Upon adding two or more graphical component objects to the canvas portion, a user may configure to link the graphical component objects, such as via a handle associated with each of the graphical component objects. A user may add additional graphical component objects and continue to link the graphical component objects via the associated handles to generate a graphical component pipeline.”)
The user can combine graphical components which may represent machine learning models via a graphical application editor. The models are used for linking inputs and outputs processed by the respective models to generate a graphical component pipeline (information processing process) for completing a task which corresponds to, accept a plurality of learned models selected by a user; and generate an information processing process by combining the plurality of learned models accepted.
Regarding claim 7, Gupte teaches the apparatus of claim 1. Gupte further teaches: wherein the processing circuitry accepts a plurality of functions selected by the user, and generates a workflow by combining the plurality of functions accepted.
([0015] “An application (e.g., a software or firmware application) as executed by or as part of one or more computing devices, may be assembled to accomplish one or more electronic tasks (which may be referred to as “tasks” in the present disclosure). Tasks may include, but not be limited to, image processing, video rendering, encoding and decoding functions, data parsing, speech recognition, natural language processing, and the like. An application may be configured to receive any number of inputs, may include one or more components (e.g., blocks of code) configured to perform tasks, such as any of the tasks described above, and may be configured to produce any number of outputs, which outputs may include results generated by the components performing the tasks.” [0023] “the component library may be displayed in the graphical application editor such that a user may be able to select one or more graphical component objects from the component library and include the selected graphical component objects in a canvas portion of the graphical application editor. Upon adding two or more graphical component objects to the canvas portion, a user may configure to link the graphical component objects, such as via a handle associated with each of the graphical component objects. A user may add additional graphical component objects and continue to link the graphical component objects via the associated handles to generate a graphical component pipeline.”)
In addition to models, components may be blocks of code for performing tasks e.g. ( encoding and decoding functions) which the user can select to generate a graphical component pipeline which corresponds to, wherein the processing circuitry accepts a plurality of functions selected by the user, and generates a workflow by combining the plurality of functions accepted.
Regarding claim 8, Gupte teaches the apparatus of claim 7. Gupte further teaches: the processing circuitry generates the information processing process including a combination in which output data of the first learned model included in the plurality of learned models serves as input data of a first function included in the plurality of functions, and an output data type of the first learned model is included in an input data type of the first function
([0041] “The component library 110 may include a repository of components that may be used to create the application 135. The components may include blocks of code configured to perform certain tasks. … Additionally, or alternatively, the components may include model components” [0029] “a first handle of a first graphical component object may be compatible with a second handle of a second graphical component object when the first handle type is the same as or includes a subset of the second handle type. For example, a first component may include an audio source with a handle associated with audio data on a single channel and a second component may include an audio decoder with a handle associated with audio data on multiple channels. The handle of the audio source may be compatible with the handle of the audio renderer as the handle of the audio source is configured to output a single channel of audio and the handle of the audio renderer is configured to receive multiple channels of audio, which includes a single channel.” [0030] “For example, in instances in which an output from a first graphical component object is linked to a second graphical component object via associated handles, the application may include the underlying code of the first component and of the second component in the application and may link an output from a first component to a second component using the data format and/or the data characteristic defined by the handles, such that a component pipeline may be included in an application.”)
The generated component pipeline comprises components and handles. Handles are used to connect components (e.g. model components and code block components) and place constraints representing the data type output by a source component and received by a target component wherein each component has one or more respective handles. The handle data type contains a data format and associated characteristic. When two components such as a model ad component 1 and a code block as component 2 are connected via their compatible/associated handles such that data output from a first model component may flow to a second code block component as input data, this corresponds to, the processing circuitry generates the information processing process including a combination in which output data of the first learned model included in the plurality of learned models serves as input data of a first function included in the plurality of functions, and an output data type of the first learned model is included in an input data type of the first function.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2,3,4 and 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gupte et al. (US 20220391176 A1) hereafter Gupte in view of Amtrup et al. (US 9483794 B2) hereafter Amtrup.
Regarding claim 2, Gupte teaches the apparatus of claim 1. Gupte further teaches: wherein the processing circuitry generates the information processing process including a combination in which output data of a first learned model included in the plurality of learned models serves as input data of a second learned model included in the plurality of learned models,
([0025] ”each graphical component object in the graphical application editor may include one or more handles associated therewith. In some embodiments, the graphical application editor may be configured to illustrate links between graphical component objects by linking handles between the graphical component objects such that the handles are illustrated as potential connection points between graphical component objects. … a graphical component object may be able to be linked with another graphical component object via any of the multiple handles… The graphical component pipelines may indicate a logical order of components (and their associated operations) that may be executed to perform a task.” [0029] “a first handle of a first graphical component object may be compatible with a second handle of a second graphical component object when the first handle type is the same as or includes a subset of the second handle type. For example, a first component may include an audio source with a handle associated with audio data on a single channel and a second component may include an audio decoder with a handle associated with audio data on multiple channels. The handle of the audio source may be compatible with the handle of the audio renderer as the handle of the audio source is configured to output a single channel of audio and the handle of the audio renderer is configured to receive multiple channels of audio, which includes a single channel.” [0030] “For example, in instances in which an output from a first graphical component object is linked to a second graphical component object via associated handles, the application may include the underlying code of the first component and of the second component in the application and may link an output from a first component to a second component using the data format and/or the data characteristic defined by the handles, such that a component pipeline may be included in an application.” [0081] “Referring back to FIG. 1 , in some embodiments, two or more components linked together may form the component pipeline 125… the arrangement of the components in the component pipeline 125 may be such that outputs from the tasks performed by the multiple components may be inputs to other components in the component pipeline 125. In these or other embodiments, the component pipeline 125 may include the graphical component objects in the graphical application editor 115, such that the component pipeline 125 may represent a component pipeline of the components that the graphical component objects represent. For example, the component pipeline 125 of the graphical application editor 115 may include a first graphical component object linked to a second graphical component object, where the output from the first graphical component object may be input to the second graphical component object, forming the component pipeline 125.“)
Graphical components may represent models, which are linked via handles for performing tasks of a generated pipeline (e.g. transferring data for performing audio processing). When the graphical application editor contains two components implemented as models linked via a handle, the link between components represented by an arrow indicates the flow output data from one component as input data to another component (e.g. the first component outputs a single channel of audio to a second component that receives the audio channel input via the handle connection), this corresponds to, wherein the processing circuitry generates the information processing process including a combination in which output data of a first learned model included in the plurality of learned models serves as input data of a second learned model included in the plurality of learned models.
and an output data constraint including an output data type and an output ([0029] “a first handle of a first graphical component object may be incompatible with a second handle of a second graphical component object when the data format and/or the data characteristic associated with the first handle (e.g., the first handle type) is not the same as or not a subset of the data format and/or the data characteristic associated with the second handle (e.g., the second handle type). In general, in instances in which an output generated by the first graphical component object and associated with a first handle includes a format that is unusable by the second graphical component object, the handle of the first graphical component object and the handle of the second graphical component object may be considered incompatible… In the alternative, a first handle of a first graphical component object may be compatible with a second handle of a second graphical component object when the first handle type is the same as or includes a subset of the second handle type.” [0030] “For example, in instances in which an output from a first graphical component object is linked to a second graphical component object via associated handles, the application may include the underlying code of the first component and of the second component in the application and may link an output from a first component to a second component using the data format and/or the data characteristic defined by the handles, such that a component pipeline may be included in an application.” [0072] “In some embodiments, the data format of the handle type may include a data structure or a data container in which data associated with the graphical component object may be presented. For example, the data format may include a video, an image, audio, numerical values, and the like, all of which may be associated with the graphical component object with which the handle is associated. In some embodiments, the data characteristic of the handle type may include variations in the arrangement of the data, such as a number of channels that may be included in the handle. For example, the data characteristic may include a single channel, multiple channels, static data, dynamic data, and the like, all of which may be associated with the graphical component object with which the handle is associated.”)
Handles are used to connect components (e.g. model components) and place constraints representing the data type output by a source component and received by a target component wherein each component has one or more respective handles. The handle data type comprises a data format field and associated characteristic field which correspond to a data type and additional input/output constraint value. When two components are connected via their compatible/associated handles such that data output from a first component may flow to a second component as input data, the associated handles underlying matching data formats correspond to, and an output data constraint including an output data type and an output of the first learned model is included in an input data constraint including an input data type and an input of the second learned model.
Gupte does not explicitly teach: data constraint including an … output roll … data constraint including.. an input roll
However Amtrup suggests: data constraint including an .. output roll … data constraint including.. an input roll
([Col 33, Line 63 - Col 34, Line 17] “classification techniques and/or workflows leveraging ID classification may involve associating one or more of the ID classification(s) with the image of the ID. Preferably, the association is or reflects a data-metadata relationship, e.g. between the image data and the classification of the ID depicted in the image data. More preferably… the data and/or metadata are stored in a manner so as to permit ease of access and utilization by one or more workflows. … the data and/or metadata may be stored in a format compatible with the workflow and/or software associated with the workflow (especially image processing software). In such approaches, exemplary inventive concepts described herein may include associating various data with other data or processes, e.g. workflow operations, etc. For example, in one embodiment the image depicting the ID may be associated with a classification to which the ID is determined to belong. Optionally, but preferably, the classification is associated with the image as metadata tag(s) applied to the image data.” [Col 30, Lines 1-39] “The method 600 also includes operation 604, in which the ID is classified. The classification may take any form such as described herein, but preferably the classifying is based at least in part on comparing feature vector data…. In operation 606, method 600 includes providing the ID and the ID classification to a workflow, preferably a workflow also instantiated on the mobile device and/or executable at least in part using the mobile device. The ID and ID classification may be provided to the workflow in any suitable manner, e.g. by storing one or more of the ID and the ID classification to memory and associating the corresponding memory locations with the workflow, by communicating with a remote data storage system storing the ID and/or ID classification, via another workflow or another operation within a same workflow, etc. as would be understood by skilled artisans reading these descriptions. With continuing reference to FIG. 6, method 600 includes operation 608, where at least a portion of the workflow is driven based at least in part on the ID and the ID classification. Preferably, the workflow is driven based on identifying information represented in the ID and/or determined using identifying information or other information (e.g. an ID identifier) obtained from the ID. The workflow is even more preferably also driven based on the ID classification, e.g. determining select workflows from among a plurality of workflows to which the ID classification relates, pertains, or is applicable.”)
Amtrup provides data to a workflow containing an image ID and classification in which the classification corresponds to a metadata constraint representative of the input/output data analogous to the instant specification’s definition of input/output roll data, which corresponds to, data constraint including an output roll and data constraint including an input roll. Therefore, it would have been obvious to one of ordinary skill in the art to which said subject matter pertains before the effective filing date of the claimed invention to combine Gupte with , data constraint including an output roll and data constraint including an input roll, as taught by Amtrup, by implementing the additional roll/classification parameter in the handle constraint data type of Gupte which define inputs and outputs passed between components; Because including additional model parameters associated with a machine learning model is a well understood implementation that is supported by the interface taught by Gupte, evidenced at [0091] and [0092]. Methods outlined by Amtrup serve to provide improvements to the workflows generated by Gupte by leveraging classifications , Amtrup [Col 35, Lines 52-54 ] “as a filter to effectively reduce computational cost of downstream operations in the workflow.” Furthermore, embodiments of the instant specification at [0018] and [0044] in which roll data classifications encompass a “document, identification card, driver's license, health insurance card”, are analogous to embodiments of Amtrup at [Col 25, Lines 59-61] “classifying the ID as a particular type of ID (e.g. driver's license, credit card, social security card, tax form, passport, military ID, employee ID, insurance card, etc. …and so on, as would be understood by one having ordinary skill in the art upon reading the present descriptions.”
Regarding claim 3, Gupte and Amtrup teach the apparatus of claim 2. Gupte further teaches: the processing circuitry generates the information processing process including a combination in which the output data of the first learned model included in the plurality of learned models serves as input data of a third learned model included in the plurality of learned models,
PNG
media_image1.png
476
614
media_image1.png
Greyscale
([0074] “In FIG. 4 , an example component pipeline 400 may include a first graphical component object 410, a second graphical component object 420, and a connecting graphical component object 430. The component pipeline 400 may be the same or similar as the component pipeline 125 of FIG. 1 . [0077] “Alternatively, or additionally, the connecting graphical component object 430 may be included in the component pipeline 400…In such instances, the first handle 412 may be linked with the source handle 432 and the target handle 434 may be linked with the second handle 422 such that the first graphical component object 410 may be linked with the second graphical component object 420 via the connecting graphical component object 430.” [0078] “additional connecting graphical component objects may be disposed between the first graphical component object 410 and the second graphical component object 420 … For example, one or more input and/or output buffers, data conditioners and/or transformative components… For example, the first graphical component object 410 may be linked to a transmitter graphical component object and the second graphical component object 420 may be linked to a receiver graphical component object such that an output from the first graphical component object 410 may be input to the second graphical component object 420 via the transmitter graphical component object and the receiver graphical component object.” [0081] “Referring back to FIG. 1 , in some embodiments, two or more components linked together may form the component pipeline 125… the arrangement of the components in the component pipeline 125 may be such that outputs from the tasks performed by the multiple components may be inputs to other components in the component pipeline 125. In these or other embodiments, the component pipeline 125 may include the graphical component objects in the graphical application editor 115, such that the component pipeline 125 may represent a component pipeline of the components that the graphical component objects represent. For example, the component pipeline 125 of the graphical application editor 115 may include a first graphical component object linked to a second graphical component object, where the output from the first graphical component object may be input to the second graphical component object, forming the component pipeline 125.“)
Graphical components may represent models, which are linked via handles for performing tasks of a generated pipeline. When the graphical application editor contains at least two components implemented as models linked via a handle, the link between components represented by an arrow indicates the flow output data from one component as input data to another component. See figure 4, which contains 3 component objects which may be implemented as models connected via arrowed lines representing the flow of output data from the first component 410 to the second component 420 and third component 430. The connection between first component 410 and third component 430 indicating data flow corresponds to, the processing circuitry generates the information processing process including a combination in which the output data of the first learned model included in the plurality of learned models serves as input data of a third learned model included in the plurality of learned models.
and the first learned model selectively outputs the output data to any one of the second learned model and the third learned model such that the output data constraint of the output data output from the first learned model is included in an input data constraint of the second learned model or the third learned model.
([0076] “ the user may update a handle depiction such that the new handle 414 may be associated with the first graphical component object 410 …, the first graphical component object 410 may be linked with the second graphical component object 420 by connecting the new handle 414 to the second handle 422 as the third handle type of the new handle 414 may be compatible with the second handle type of the second handle 422.” [0077] “Alternatively, or additionally, the connecting graphical component object 430 may be included in the component pipeline 400…In such instances, the first handle 412 may be linked with the source handle 432 and the target handle 434 may be linked with the second handle 422 such that the first graphical component object 410 may be linked with the second graphical component object 420 via the connecting graphical component object 430.”)
Graphical components may represent models, which are linked via handles for performing tasks of a generated pipeline. When the graphical application editor contains at least two components implemented as models linked via a handle, the link between components represented by an arrow which indicates the flow output data from one component as input data to another component. Handles place constraints representing the data type output by a source component and received by a target component wherein each component has one or more respective handles that are required to be compatible in order to transfer data i.e. the output handle data type (constraint) of first component 410 must be compatible with the input handle data types of second component 420 and third component 430 as indicated by the connecting lined between the components. In figure 4 depending on the user’s desired implementation, the first component 410 may implement a new handle for transferring data directly to second component 420 or may transfer data to third component 430 which corresponds to, and the first learned model selectively outputs the output data to any one of the second learned model and the third learned model such that the output data constraint of the output data output from the first learned model is included in an input data constraint of the second learned model or the third learned model.
Regarding claim 4, Gupte and Amtrup teach the apparatus of claim 3. Gupte further teaches:
wherein the first learned model: outputs the output data to the second learned model when the output data constraint of the output data output from the first learned model is included in the input data constraint of the second learned model;
([0073] “In instances in which a handle type associated with the first handle 312 of the first graphical component object 310 is compatible with a handle type associated with the second handle 322 of the second graphical component object 320, a user may link the first graphical component object 310 with the second graphical component object 320 by connecting the first handle 312 to the second handle 322 as part of the component pipeline 300. The user may link the first graphical component object 310 with the second graphical component object 320 in a graphical application editor, such as the graphical application editor 115 of FIG. 1 . In some embodiments, the first handle 312 may be compatible with the second handle 322 when the first handle type is the same as or includes a subset of the second handle type. “ [0081] “Referring back to FIG. 1 , in some embodiments, two or more components linked together may form the component pipeline 125. In some embodiments, the arrangement of the components in the component pipeline 125 may be such that outputs from the tasks performed by the multiple components may be inputs to other components in the component pipeline 125. For example, the component pipeline 125 of the graphical application editor 115 may include a first graphical component object linked to a second graphical component object, where the output from the first graphical component object may be input to the second graphical component object, forming the component pipeline 125.”)
Graphical components may represent models. Handles place constraints representing the data type output by a source component and received by a target component wherein each component has one or more respective handles that are required to be compatible in order to transfer data i.e. the output handle data type (constraint) of first component must be compatible with the input handle data types of second component. When the first component linked to the second component via a compatible handle sends data, the transfer corresponds to, wherein the first learned model: outputs the output data to the second learned model when the output data constraint of the output data output from the first learned model is included in the input data constraint of the second learned model.
does not output the output data to the second learned model when the output data constraint of the output data output from the first learned model is not included in the input data constraint of the second learned model;
([0074] “The component pipeline 400 may be the same or similar as the component pipeline 125 of FIG. 1 . In FIG. 4 , the first handle 412 of the first graphical component object 410 may include a first handle type that may be incompatible with a second handle type of the second handle 422 of the second graphical component object 420. In instances in which a handle type associated with the first graphical component object 410 is incompatible with a handle type associated with the second graphical component object 420, a graphical application editor associated with the component pipeline 400, such as the graphical application editor 115 of FIG. 1 , may provide an indication to the user that the first handle 412 may not be linked with the second handle 422.” [0075] “In some embodiments, a first handle type of the first handle 412 may be incompatible with a second handle type of the second handle 422 when the data format and/or the data characteristic associated with the first handle 412 (e.g., the first handle type) is not the same as or not a subset of the data format and/or the data characteristic associated with the second handle 422 (e.g., the second handle type). In general, in instances in which an output generated by a first component represented by the first graphical component object 410 and associated with the first handle 412 includes a format that is unusable by a second component represented by the second graphical component object 420, the first handle type of the first handle 412 and the second handle type of the second handle 422 may be considered incompatible.”)
See also par [0024]. Components representing models may be linked, however handles which define input/output constraints of respective components may be incompatible preventing the transfer or data between 2 components, which corresponds to, does not output the output data to the second learned model when the output data constraint of the output data output from the first learned model is not included in the input data constraint of the second learned model;
outputs the output data to the third learned model when the output data constraint of the output data output from the first learned model is included in the input data constraint of the third learned model;
([0074] “In FIG. 4 , an example component pipeline 400 may include a first graphical component object 410, a second graphical component object 420, and a connecting graphical component object 430. The component pipeline 400 may be the same or similar as the component pipeline 125 of FIG. 1 . [0077] “Alternatively, or additionally, the connecting graphical component object 430 may be included in the component pipeline 400…In such instances, the first handle 412 may be linked with the source handle 432 and the target handle 434 may be linked with the second handle 422 such that the first graphical component object 410 may be linked with the second graphical component object 420 via the connecting graphical component object 430.” [0078] “additional connecting graphical component objects may be disposed between the first graphical component object 410 and the second graphical component object 420 … For example, one or more input and/or output buffers, data conditioners and/or transformative components… For example, the first graphical component object 410 may be linked to a transmitter graphical component object and the second graphical component object 420 may be linked to a receiver graphical component object such that an output from the first graphical component object 410 may be input to the second graphical component object 420 via the transmitter graphical component object and the receiver graphical component object.”) Data travels through components connected with compatible handles. When the first component is connected to the third connecting component in figure 4 through a compatible handle, the transfer of data between the components corresponds to, outputs the output data to the third learned model when the output data constraint of the output data output from the first learned model is included in the input data constraint of the third learned model.
and does not output the output data to the third learned model when the output data constraint of the output data output from the first learned model is not included in the input data constraint of the third learned model.
([0074] “The component pipeline 400 may be the same or similar as the component pipeline 125 of FIG. 1 . In FIG. 4 , the first handle 412 of the first graphical component object 410 may include a first handle type that may be incompatible with a second handle type of the second handle 422 of the second graphical component object 420. In instances in which a handle type associated with the first graphical component object 410 is incompatible with a handle type associated with the second graphical component object 420, a graphical application editor associated with the component pipeline 400, such as the graphical application editor 115 of FIG. 1 , may provide an indication to the user that the first handle 412 may not be linked with the second handle 422.” [0075] “In some embodiments, a first handle type of the first handle 412 may be incompatible with a second handle type of the second handle 422 when the data format and/or the data characteristic associated with the first handle 412 (e.g., the first handle type) is not the same as or not a subset of the data format and/or the data characteristic associated with the second handle 422 (e.g., the second handle type). In general, in instances in which an output generated by a first component represented by the first graphical component object 410 and associated with the first handle 412 includes a format that is unusable by a second component represented by the second graphical component object 420, the first handle type of the first handle 412 and the second handle type of the second handle 422 may be considered incompatible.”)
See also par [0024]. Components representing models may be linked, however handles which define input/output constraints of respective components may be incompatible preventing the transfer or data between 2 components, which corresponds to, and does not output the output data to the third learned model when the output data constraint of the output data output from the first learned model is not included in the input data constraint of the third learned model.
Regarding claim 9, Gupte teaches the apparatus of claim 7. Gupte further teaches: wherein the processing circuitry presents, to the user, a plurality of functions that are selectable by the user,
([0023] “the component library may be displayed in the graphical application editor such that a user may be able to select one or more graphical component objects from the component library and include the selected graphical component objects in a canvas portion of the graphical application editor. Upon adding two or more graphical component objects to the canvas portion, a user may configure to link the graphical component objects, such as via a handle associated with each of the graphical component objects. A user may add additional graphical component objects and continue to link the graphical component objects via the associated handles to generate a graphical component pipeline.”)
As stated in the rejection of claim 7, components of the library may be graphical representations of code implementing specific application functions. The displayed component library allows users to select components, which corresponds to, wherein the processing circuitry presents, to the user, a plurality of functions that are selectable by the user.
based on input data constraints including input data types and input
([0024] ”Some graphical component objects of the component library used by a graphical application editor may be incompatible with other graphical component objects in the graphical application editor. For example, in some circumstances, the incompatibility may arise when a first graphical component object includes a first interface (e.g., such as a first defined standard) and a second graphical component object includes a second interface (e.g., such as a second defined standard… In instances in which two components implement different standards, data generated and output by the first component may not be received as input by the second component as the interfaces may be incompatible. In practice, when attempting to connect incompatible handles of different graphical components, an indication may be provided (e.g., visually, audibly, etc.) to the user to indicate the incompatibility. For example, a connection may not be allowed for incompatible components, or a connection may be highlighted in a certain color or indicated with a symbol (e.g., an “x”) indicating that the connection is in error.” [0080] “in some embodiments, the graphical application editor may be configured to provide recommendations in response to the user attempting to connect incompatible handle types. For instance, the graphical application editor may be configured to analyze different handle types that are associated with the graphical component objects that are being attempted to be linked to determine whether any of such handle types are compatible. In response to identifying compatible handle types, the graphical application editor may suggest using the identified handle types, and/or may automatically adjust the handle linking to include compatible handles. Additionally, or alternatively, the graphical application editor may be configured to similarly analyze whether any other components may have handle types that would allow a corresponding component to be used as a connecting component.”)
Handles defined by a handle data type contain a data format and associated characteristic. When a user selects and adds a first component to the workspace and attempts to adds a second component , the system gives a graphical indication based on the compatibility between the handles of the existing component in the workspace and the component the user wants to add indicating if the component is a valid selection. The user selecting a compatible second component to add to the pipeline based on the indication corresponds to, based on input data constraints including input data types and input of the plurality of learned models accepted or output data constraints including output data types and output of the plurality of learned models, and accept a plurality of functions selected by the user from among the plurality of functions selectable by the user that are presented.
Gupte does not explicitly teach: based on input data constraints including … input rolls …or output data constraints including … output rolls
However Amtrup suggests: based on input data constraints including … input rolls …or output data constraints including … output rolls
([Col 33, Line 63 - Col 34, Line 17] “classification techniques and/or workflows leveraging ID classification may involve associating one or more of the ID classification(s) with the image of the ID. Preferably, the association is or reflects a data-metadata relationship, e.g. between the image data and the classification of the ID depicted in the image data. More preferably… the data and/or metadata are stored in a manner so as to permit ease of access and utilization by one or more workflows. … the data and/or metadata may be stored in a format compatible with the workflow and/or software associated with the workflow (especially image processing software). In such approaches, exemplary inventive concepts described herein may include associating various data with other data or processes, e.g. workflow operations, etc. For example, in one embodiment the image depicting the ID may be associated with a classification to which the ID is determined to belong. Optionally, but preferably, the classification is associated with the image as metadata tag(s) applied to the image data.” [Col 30, Lines 1-39] “The method 600 also includes operation 604, in which the ID is classified. The classification may take any form such as described herein, but preferably the classifying is based at least in part on comparing feature vector data…. In operation 606, method 600 includes providing the ID and the ID classification to a workflow, preferably a workflow also instantiated on the mobile device and/or executable at least in part using the mobile device. The ID and ID classification may be provided to the workflow in any suitable manner, e.g. by storing one or more of the ID and the ID classification to memory and associating the corresponding memory locations with the workflow, by communicating with a remote data storage system storing the ID and/or ID classification, via another workflow or another operation within a same workflow, etc. as would be understood by skilled artisans reading these descriptions. With continuing reference to FIG. 6, method 600 includes operation 608, where at least a portion of the workflow is driven based at least in part on the ID and the ID classification. Preferably, the workflow is driven based on identifying information represented in the ID and/or determined using identifying information or other information (e.g. an ID identifier) obtained from the ID. The workflow is even more preferably also driven based on the ID classification, e.g. determining select workflows from among a plurality of workflows to which the ID classification relates, pertains, or is applicable.”)
Amtrup provides data to a workflow containing an image ID and classification in which the classification corresponds to a metadata constraint representative of the input/output data analogous to the instant specification’s definition of roll data, which corresponds to, based on input data constraints including input rolls or output data constraints including output rolls.
Therefore, it would have been obvious to one of ordinary skill in the art to which said subject matter pertains before the effective filing date of the claimed invention to combine Gupte with , based on input data constraints including input rolls or output data constraints including output rolls, as taught by Amtrup, by implementing the additional roll/classification parameter in the handle constraints of Gupte which define inputs and outputs passed between components; Because including additional model parameters associated with a machine learning model is a well understood implementation that is supported by the interface taught by Gupte, evidenced at [0091] and [0092]. Methods outlined by Amtrup serve to provide improvements to the workflows generated by Gupte by leveraging classifications , Amtrup [Col 35, Lines 52-54 ] “as a filter to effectively reduce computational cost of downstream operations in the workflow.” Furthermore, embodiments of the instant specification at [0018] and [0044] in which roll data classifications encompass a “document, identification card, driver's license, health insurance card”, are analogous to embodiments of Amtrup at [Col 25, Lines 59-61] “classifying the ID as a particular type of ID (e.g. driver's license, credit card, social security card, tax form, passport, military ID, employee ID, insurance card, etc. …and so on, as would be understood by one having ordinary skill in the art upon reading the present descriptions.”
Regarding claim 10, it is a non-transitory computer-readable storage medium claim having similar limitations to those cited in the rejection of claim 1. Thus claim 10 is also rejected under the same rationale as cited in the rejection of claim 1 above.
Regarding claim 11, it is a method claim having similar limitations to those cited in the rejection of claim 1. Thus claim 11 is also rejected under the same rationale as cited in the rejection of claim 1 above.
Claim 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gupte et al. (US 20220391176 A1) hereafter Gupte in view of (US 11675605 B2) hereafter Chakraborty.
Regarding claim 5, Gupte teaches the apparatus of claim 1. Gupte further teaches:
([0018] “Combining components to perform a task may include linking graphical component objects in a graphical application editor such that an application may be generated by the graphical application editor by arranging the corresponding components and their respective underlying code based on the arrangement of the graphical component objects. The linked graphical component objects may be illustrative of the linking between components (e.g., the inputs and outputs of the corresponding components) as a component pipeline.” [0019] “Some components that may be visually represented in the graphical application editor may be configured as a machine learning model, or simply “model” in the present disclosure, unless otherwise defined.”)
The user can combine graphical components which may represent machine learning models via a graphical application editor. The models are used for linking inputs and outputs processed by the respective models to generate a graphical component pipeline (information processing process) for completing a task which corresponds to, and generates the information processing process by combining the plurality of learned models accepted.
Gupte does not explicitly teach: wherein the processing circuitry accepts a plurality of screens selected by the user, and generates the information processing process by combining the plurality of learned models accepted and the plurality of screens accepted.
However Chakraborty teaches: wherein the processing circuitry accepts a plurality of screens selected by the user, and generates the information processing process by combining the plurality of learned models accepted and the plurality of screens accepted.
([Col 7, Lines 43 – 54] “ embodiments described herein provide a data pipeline configuration system that allows data pipelines to be configured using an intuitive visual interface. The pipeline configuration system allows graphical pipeline components representing data sources, data processing, analytic and machine learning (ML) models, and emitters to be selectively added to a data pipeline application by selecting these components from a preconfigured library, also referred to as a palette. The pipeline application is created by arranging and linking these selected pipeline components within a graphical development interface rendered by the system.” [Col 23, Lines 40-68]“The data emitter component can be selected from the component library made available by the pipeline configuration system, and can represent a specified type of data sink or destination to which output data from the pipeline is to be published (e.g., … reporting application, a messaging or notification system… If such a data emitter component is selected (YES at step 1734), the methodology proceeds to step 1736, where the data emitter component selected at step 1734 is added to the pipeline application. … This emitter configuration input maps selected items of pipeline output data to a data sink entity represented by the selected data emitter component. Pipeline data that can be mapped in this manner can include raw or processed data from the data source as well as analytic results generated by any models added to the pipeline application .”)
Combining emitter/data sink components representing a reporting application or messaging notification system with model components to generate a pipeline application corresponds to, wherein the processing circuitry accepts a plurality of screens selected by the user, and generates the information processing process by combining the plurality of learned models accepted and the plurality of screens accepted.
Therefore, it would have been obvious to one of ordinary skill in the art to which said subject matter pertains before the effective filing date of the claimed invention to combine Gupte with, wherein the processing circuitry accepts a plurality of screens selected by the user, and generates the information processing process by combining the plurality of learned models accepted and the plurality of screens accepted, as taught by Chakraborty; Because including screens in a generated pipeline allows users to map data inputs/outputs to compatible user facing interfaces, thereby streamlining the workflow to reduce friction associated with developers having to manually configure data source/sink interfaces as evidenced by Chakraborty [Col 18, Lines 15-54].
Claim 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Gupte et al. (US 20220391176 A1) hereafter Gupte in view of (US 11675605 B2) hereafter Chakraborty and further in view of Amtrup et al. (US 9483794 B2) hereafter Amtrup.
Regarding claim 6, Gupte teaches the apparatus of claim 1. Gupte further teaches:
wherein the processing circuitry generates the information processing process including a combination in which output data
( [0030] “For example, in instances in which an output from a first graphical component object is linked to a second graphical component object via associated handles, the application may include the underlying code of the first component and of the second component in the application and may link an output from a first component to a second component using the data format and/or the data characteristic defined by the handles, such that a component pipeline may be included in an application.” [0081] “Referring back to FIG. 1 , in some embodiments, two or more components linked together may form the component pipeline 125… the arrangement of the components in the component pipeline 125 may be such that outputs from the tasks performed by the multiple components may be inputs to other components in the component pipeline 125. In these or other embodiments, the component pipeline 125 may include the graphical component objects in the graphical application editor 115, such that the component pipeline 125 may represent a component pipeline of the components that the graphical component objects represent. For example, the component pipeline 125 of the graphical application editor 115 may include a first graphical component object linked to a second graphical component object, where the output from the first graphical component object may be input to the second graphical component object, forming the component pipeline 125.“) Components may be implemented as models. The generated component pipeline where output from a first component serves as input to a second component, corresponds to, wherein the processing circuitry generates the information processing process including a combination in which output data serves as input data of the first learned model included in the plurality of learned models;
and specifies, based on an input data constraint including an input data type and input
([0030] “For example, in instances in which an output from a first graphical component object is linked to a second graphical component object via associated handles, the application may include the underlying code of the first component and of the second component in the application and may link an output from a first component to a second component using the data format and/or the data characteristic defined by the handles, such that a component pipeline may be included in an application.” [0072] “In some embodiments, the data format of the handle type may include a data structure or a data container in which data associated with the graphical component object may be presented. For example, the data format may include a video, an image, audio, numerical values, and the like, all of which may be associated with the graphical component object with which the handle is associated. In some embodiments, the data characteristic of the handle type may include variations in the arrangement of the data, such as a number of channels that may be included in the handle. For example, the data characteristic may include a single channel, multiple channels, static data, dynamic data, and the like, all of which may be associated with the graphical component object with which the handle is associated.”)
Handles are used to connect components (e.g. model components) and place constraints representing the data type output by a source component and received by a target component wherein each component has one or more respective handles. The handle data type contains a data format and associated characteristic which correspond to a data type and additional input/output constraint value. When two components are connected via their compatible/associated handles such that data output from a first component may flow to a second component as input data, the associated handles underlying matching data formats correspond to, and specifies, based on an input data constraint including an input data type and input of the first learned model, an input field included in a first input and a data constraint including a data type and
Gupte does not explicitly teach: output data of a first screen included in the plurality of screens serves as input data of the first learned model included in the plurality of learned models;
However Chakraborty teaches: output data of a first screen included in the plurality of screens serves as input data of the first learned model included in the plurality of learned models;… an input field included in a first input screen
([Col 7, Lines 43 – 54] “ embodiments described herein provide a data pipeline configuration system that allows data pipelines to be configured using an intuitive visual interface. The pipeline configuration system allows graphical pipeline components representing data sources, data processing, analytic and machine learning (ML) models, and emitters to be selectively added to a data pipeline application by selecting these components from a preconfigured library, also referred to as a palette. The pipeline application is created by arranging and linking these selected pipeline components within a graphical development interface rendered by the system.” [Col 6, Lines 16-43] “Industrial automation systems often include one or more human-machine interfaces (HMIs) 114 that allow plant personnel to view telemetry and status data associated with the automation systems, and to control some aspects of system operation. HMIs 114 may communicate with one or more of the industrial controllers 118 over a plant network 116, and exchange data with the industrial controllers to facilitate visualization of information relating to the controlled industrial processes on one or more pre-developed operator interface screens. HMIs 114 can also be configured to allow operators to submit data to specified data tags or memory addresses of the industrial controllers 118, thereby providing a means for operators to issue commands to the controlled systems (e.g., cycle start commands, device actuation commands, etc.), to modify setpoint values, etc. HMIs 114 can generate one or more display screens through which the operator interacts with the industrial controllers 118, and thereby with the controlled processes and/or systems.“[Col 8, Lines 39-49] “Input data that can be received via user interface component 204 can include, but is not limited to, pipeline design input that selects and configures pipeline components and analytic models for inclusion in the pipeline, mapping input that maps selected data items to input
fields of a selected model, or other such input data. Output data rendered by user interface component 204 can include, but is not limited to, pipeline components and models that can be selectively integrated into a data pipeline configuration, parameters of analytic or machine learning models, model scoring results, or other such output data. [Col 11, Lines 11-40] “Pipeline components 222 can represent various types of entities, processing, analytics, or ML model applications that the user wishes to include in the pipeline. In a typical pipeline application, the left-most components in the pipeline representation can be a data source component 222 a representing a data source for the data 302 that will be batched or streamed through the pipeline. The data source can be an industrial device (e.g., an industrial controller, a variable frequency drive, etc.), a data historian, a file system from which data is retrieved, an edge device (e.g., edge device 304) that collects input data and places the data on the pipeline, one or more industrial devices operating in a plant facility (e.g., industrial controllers, motor drives, sensors, telemetry devices, etc.), another application that generates data to be placed on the pipeline (e.g., via native connectivity),…The data from any of these data sources is ingested into the memory of pipeline configuration system 202 for further use by succeeding data pipeline components. The component library 406 can include a variety of data source components 222 a representing different types of data sources, which can be selectively added to the pipeline design and configured to map to the user's data source.”)
Generating a component pipeline wherein components include HMI screens allowing user input data fields to be connected to ML model applications as input corresponds to, output data of a first screen included in the plurality of screens serves as input data of the first learned model included in the plurality of learned models;… an input field included in a first input screen.
Therefore, it would have been obvious to one of ordinary skill in the art to which said subject matter pertains before the effective filing date of the claimed invention to combine Gupte with, wherein the processing circuitry generates the information processing process including a combination in which output data of a first screen included in the plurality of screens serves as input data of the first learned model included in the plurality of learned models; Because including screens in a generated pipeline allows users to map data inputs/outputs to compatible user facing interfaces, thereby streamlining the workflow to reduce friction associated with developers having to manually configure data source/sink interfaces as evidenced by Chakraborty [Col 18, Lines 15-54].
Gupte and Chakraborty do not explicitly teach: specifies, based on an input data constraint including an input data type and input roll… a data constraint including a data type and roll of the input field
However Amtrup teaches: specifies, based on an input data constraint including an input data type and input roll… a data constraint including a data type and roll of the input field
([Col 32, Lines 4-11] “a predetermined stimulus in the context of a workflow configured to facilitate a financial transaction may include attempting an operation …displaying a user interface comprising one or more fields relating to requiring or configured to receive, store, etc. an entity's financial information.” [Col 33, Line 63 - Col 34, Line 17] “classification techniques and/or workflows leveraging ID classification may involve associating one or more of the ID classification(s) with the image of the ID. Preferably, the association is or reflects a data-metadata relationship, e.g. between the image data and the classification of the ID depicted in the image data. More preferably… the data and/or metadata are stored in a manner so as to permit ease of access and utilization by one or more workflows. … the data and/or metadata may be stored in a format compatible with the workflow and/or software associated with the workflow (especially image processing software). In such approaches, exemplary inventive concepts described herein may include associating various data with other data or processes, e.g. workflow operations, etc. For example, in one embodiment the image depicting the ID may be associated with a classification to which the ID is determined to belong. Optionally, but preferably, the classification is associated with the image as metadata tag(s) applied to the image data.” [Col 30, Lines 1-39] “The method 600 also includes operation 604, in which the ID is classified. The classification may take any form such as described herein, but preferably the classifying is based at least in part on comparing feature vector data…. In operation 606, method 600 includes providing the ID and the ID classification to a workflow, preferably a workflow also instantiated on the mobile device and/or executable at least in part using the mobile device. The ID and ID classification may be provided to the workflow in any suitable manner, e.g. by storing one or more of the ID and the ID classification to memory and associating the corresponding memory locations with the workflow, by communicating with a remote data storage system storing the ID and/or ID classification, via another workflow or another operation within a same workflow, etc. as would be understood by skilled artisans reading these descriptions. With continuing reference to FIG. 6, method 600 includes operation 608, where at least a portion of the workflow is driven based at least in part on the ID and the ID classification. Preferably, the workflow is driven based on identifying information represented in the ID and/or determined using identifying information or other information (e.g. an ID identifier) obtained from the ID. The workflow is even more preferably also driven based on the ID classification, e.g. determining select workflows from among a plurality of workflows to which the ID classification relates, pertains, or is applicable.”)
Amtrup provides data to a workflow containing an image ID and classification in which the classification corresponds to a metadata constraint representative of the input/output data, analogous to the instant specification’s definition of roll data. When the classification is included in the handle constraint parameters implemented by Gupte, this corresponds to, specifies, based on an input data constraint including an input data type and input roll… a data constraint including a data type and roll of the input field. Therefore, it would have been obvious to one of ordinary skill in the art to which said subject matter pertains before the effective filing date of the claimed invention to combine Gupte and Chakraborty with , specifies, based on an input data constraint including an input data type and input roll and a data constraint including a data type and roll of the input field, as taught by Amtrup, by implementing the additional roll/classification parameter in the handle constraints of Gupte which define inputs and outputs passed between components; Because including additional model parameters associated with a machine learning model is a well understood implementation that is supported by the interface taught by Gupte, evidenced at Gupte [0091] and [0092]. Methods outlined by Amtrup serve to provide improvements to the workflows generated by Gupte by leveraging classifications , “as a filter to effectively reduce computational cost of downstream operations in the workflow.” Amtrup [Col 35, Lines 52-54 ]
Prior Art Made of Record
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
(US 20230267366 A1) [Abstract] “Systems, methods, and other embodiments associated with integrating a machine learning model into a target application are described. In one embodiment, a method includes parsing a definition file that represents the selected ML model to (1) identify one or more input features defined for the ML model, and (2) identify one or more output predictions that the ML model is configured to generate. The input features and output predictions are mapped to locations within the target application.” Relevant to one or more claims.
(US 20220300860 A1) [Abstract] “The methods and systems provide an ensemble approach that combines multiple single-model-solutions to produce optimal forward-looking forecasts.” Relevant to one or more claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LAWRENCE O'CONNOR EMANUEL whose telephone number is (571)272-8975. The examiner can normally be reached M-F 9:00 - 5:00 pm .
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chat Do can be reached at (571) 272-3721. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-919 7 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/L.S.O./ Examiner, Art Unit 2193
/Chat C Do/Supervisory Patent Examiner, Art Unit 2193
Prosecution Insights