DETAILED ACTION
This action is in response to the claims filed May 26, 2026. Claims 1-19 and 21 are pending. Claims 1, 9, and 17 are independent claims. Claims 1, 3-5, 8-9, 11-13, 19-17, 19, and 21 have been amended. Claim 20 has been cancelled.
The rejection under 35 U.S.C. 101 directed to abstract idea has been maintained in view of Applicant’s arguments and amendments to the claims.
The rejection under 35 U.S.C. 101 directed to non-statutory subject matter is maintained.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-19 and 21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claims 1, 9, and 17, the limitations “convert an image of the diagram into a description of the software architecture”, “identify a modification to be made to the software architecture and generate a textual description of the modification”, and “modify the diagram of the software architecture based on the textual description of the modification” as drafted, are functions that, under its broadest reasonable interpretation, recite the abstract idea of a mental process. The limitation encompasses a human mind carrying out the function through observation, evaluation, judgement, and/or opinion, or even with the aid of pen and paper. Thus, these limitations recite and call under the “Mental Processes” grouping of abstract ideas under Prong 1.
Under Prong 2, this judicial exception is not integrated into a practical application. The additional limitations “An apparatus comprising: a memory configured to store software architecture documents which include text-based descriptions mapped to diagrams of architecture components; and a processor coupled to the memory configured to”, “a computer-readable medium comprising instructions stored therein which when executed by a processor cause a computer to perform”, “… using a first modality of a multimodal artificial intelligence (AI) model which operates on image data”, and “…using a second modality of the multimodal AI model which operates on text data” are recited at a high-level of generality such that they amount to no more than mere instructions to apply the exception using a generic computer, and/or mere computer components. See MPEP 2106.05(f). The limitation “receive an input comprising a diagram of a software architecture implemented by a computing platform” does nothing more than add the insignificant extra solution activity of merely gathering and transmitting data to the judicial exception. See 2106.05(g). The limitation “output the modified diagram on a user interface” does nothing more than add the insignificant post solution activity of merely transmitting data to a user interface and displaying, to the judicial exception. See 2106.05(g). Accordingly, the additional elements do not integrate the recited judicial exception into a practical application, and the claim is therefore directed to the judicial exception.
Under Step 2B, the claims to not recite additional elements that re sufficient to amount to significantly more. As discussed above with respect to integration of the abstract idea into a practical application, the limitations “An apparatus comprising: a memory configured to store software architecture documents which include text-based descriptions mapped to diagrams of architecture components; and a processor coupled to the memory configured to”, “a computer-readable medium comprising instructions stored therein which when executed by a processor cause a computer to perform”, “… using a first modality of a multimodal artificial intelligence (AI) model which operates on image data”, and “…using a second modality of the multimodal AI model which operates on text data” amount to no more than mere instructions, or generic computer/computer components to carry out the exception. Mere instructions to apply an exception cannot provide an inventive concept. See MPEP 2106.05(d). For the limitations, “receive an input comprising a diagram of a software architecture implemented by a computing platform” and “output the modified diagram on a user interface”, the courts have identified mere data gathering and transmitting as well-understood, routine, and conventional activity. See MPEP 2106.05(d). Accordingly, the claims are not patent eligible under 35 U.S.C. 101.
Regarding claims 2, 10, and 18, the limitation “wherein the software architecture documents comprise a combination of diagrams and descriptions of a plurality of different software architectures” merely describes the software architecture documents recited in the mental step of claims 1, 9, and 17 and thus are part of the mental step. The same generic computer/component components are recited as in claims 1, 9, and 17 which does not amount to practical application under Prong 2, nor to significantly more under Step 2B.
Regarding claims 3, 11, and 19, the limitation “generate a diagram of a missing component from the software architecture based on…” is an additional mental step. The limitation “…based on the multimodal AI model” merely applies a generic computer component to the abstract idea of generating the description, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B as discussed above. The limitation “display the diagram via the user interface” amounts to nothing more than the insignificant post-solution activity of mere data transmission to a display or user interface, which does not amount to practical application under Prong 2, nor amount to significantly more under Step 2B, as discussed above.
Regarding claims 4 and 12, the limitation “modify the diagram based on…the sequence of prompts and the sequences of responses” is an additional mental step. The limitation “…based on the execution of the multimodal AI model” merely applies a generic computer component to the abstract idea of generating the description, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above. The limitations “display a sequence of prompts via the user interface” and “receive a sequence of responses to the sequence of prompts via the user interface” amounts to the insignificant extra solution activity of merely gathering and transmitting data, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above.
Regarding claims 5 and 13, the limitation “generate a description of a new software component to add to the software architecture” is an additional mental step. The limitation “display the description of the new software component via the user interface” amounts to nothing more than the insignificant post-solution activity of mere data transmission to a display, which does not amount to practical application under Prong 2, nor amount to significantly more under Step 2B, as discussed above. The same generic computer/computer components are recited as in claims 1 and 9 which do not amount to practical application under Prong 2, nor to significantly more under Step 2B as explained above.
Regarding claims 6 and 14, the limitation “generate source code for the new software component based on the description of the new software component” is an additional mental step. The limitation “output the source code via the user interface” amounts to nothing more than the insignificant post-solution activity of mere data transmission to a display, which does not amount to practical application under Prong 2, nor amount to significantly more under Step 2B, as discussed above. The same generic computer/computer components are recited as in claims 1 and 9 which do not amount to practical application under Prong 2, nor to significantly more under Step 2B as explained above.
Regarding claims 7 and 15, the limitation “generate a description of a modification to an existing software component within the software architecture” is an additional mental step. The limitation “display the description of the modification via the user interface” amounts to nothing more than the insignificant post-solution activity of mere data transmission to a display, which does not amount to practical application under Prong 2, nor amount to significantly more under Step 2B, as discussed above. The same generic computer/computer components are recited as in claims 1 and 9 which do not amount to practical application under Prong 2, nor to significantly more under Step 2B as explained above.
Regarding claims 8 and 16, the limitation “is configured to convert text within the digital document into an encoding prior to inputting the encoded text into the multimodal AI model” is an additional mental step. The limitation “wherein the input comprises a digital document that includes the diagram” merely further describes the “input” of the data gathering and transmission step of claims 1 and 9, and thus amounts to no more than merely gathering and transmitting data, which does not amount to practical application under Prong 2, nor amount to significantly more under Step 2B, as discussed above. The same generic computer/computer components are recited as in claims 1 and 9 which do not amount to practical application under Prong 2, nor to significantly more under Step 2B as explained above.
Regarding claim 21, no additional mental steps are recited. The limitation “the processor is configured to capture runtime data of the architecture components operating on the computing platform” amounts to the insignificant extra solution activity of merely gathering and transmitting data, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above. The limitation “further train the multimodal AI model to understand connections by executing the multimodal AI model on the runtime data of the architecture components” merely applies a generic computer component to the abstract idea of understanding component connections based on runtime data of architecture components, which does not amount to practical application under Prong 2, nor to significantly more under Step 2B, as discussed above
Claims 17-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 17 is directed to “a computer-readable medium”. However, it is noted that the specification does not provide an explicit definition of what constitutes a “a computer-readable medium”. The broadest reasonable interpretation of a claim drawn to a “computer-readable medium” typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of “computer-readable medium”, particularly when the specification is silent. See MPEP § 2111.01. When the broadest reasonable interpretation of a claim covers a signal per se, the claim must be rejected under 35 US.C. § 101 as covering non-statutory subject matter. See In re Nuijten, 500 F.3d 1346, 1356-57 (Fed. Cir. 2007) (transitory embodiments are not directed to statutory subject matter) and Interim Examination Instructions for Evaluating Subject Matter Eligibility Under 35 U.S.C. § 101, Aug. 24, 2009; p. 2. Therefore, the claimed “computer-readable medium” is ineligible subject matter under § 101. Applicant is advised to amend the claim to recite “non-transitory computer-readable storage medium” in order to overcome the 35 U.S.C. § 101 rejection.
Claims 18 and 19 depend on Claim 17 and do not cure the deficiency of Claim 17. Therefore, Claims 18 and 19 are rejected for the same reason set forth in the rejection of Claim 17.
Allowable Subject Matter
Claims 1-19 and 21 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action.
Reasons for Allowance
The following is an examiner’s statement of reasons for allowance:
The cited prior art taken along or in combination fail to teach, with the other claimed limitations, “convert an image of the diagram into a description of the software architecture using a first modality of a multimodal artificial intelligence (AI) model which operates on image data” and “identify a modification to be made to the software architecture and generate a textual description of the modification, using a second modality of the multimodal AI model with operates on the text data” as recited in claims 1, 9, and 17.
The closest cited prior art, the combination of US 20230161561 A1 (hereinafter “Ghosh”) and US 12321700 B2 (hereinafter “Akbari”), teaches using an AI to identify missing components of a software architecture diagram and modifying the program. Akbari discloses using a multimodal AI model to generate neural network architectures. However, the combination of Ghosh and Akbari fails to disclose “convert an image of the diagram into a description of the software architecture using a first modality of a multimodal artificial intelligence (AI) model which operates on image data” and “identify a modification to be made to the software architecture and generate a textual description of the modification, using a second modality of the multimodal AI model with operates on the text data”. While Ghosh discloses generation of a diagram description and identification of a missing component, it does not generate a textual description using an AI model operating on text data, nor does it use a multimodal AI model. Akbari discloses using a multimodal AI model to generate neural network architectures but does not train on image data.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Response to Arguments
Applicant's arguments filed May 26, 2026, have been fully considered but they are not persuasive.
In the remarks, Applicant argues:
Applicant points to the USPTO's "2019 Revised Patent Subject Matter Eligibility Guidance," (2019 Guidance) which took effect on January 7, 2019. The 2019 Guidance, provides a 2-prong inquiry into "step 2A of the USPTO's Subject Matter Eligibility Guidance as incorporated into the Manual of Patent Examining Procedure ("MPEP") 2106." Page 4 of the 2019 Guidance.
The first prong (2A1) of the 2019 Guidance requires that the Examiner identifies which claim limitation(s) are abstract. In order for a claim limitation to be abstract, the claim limitation has to be one of a mathematical concept, a certain method of organizing human activity, or a mental process. "Claims that do not recite matter that falls within these enumerated groupings of abstract ideas should not be treated as reciting abstract ideas," thus making the claims patent eligible. See, Page 11 of the 2019 Guidance.
MPEP 2106.04(II) notes the claim should be considered "as a whole" when performing the analysis under Prong I of the Step 2A analysis. Applicant respectfully submits that when Claim 1 is viewed "as a whole", Claim 1 is not directed to an abstract concept, but rather a process of a multimodal AI model operating on different machine-processing modalities associated with different data types. In particular, in Claim 1, an image of a software architecture diagram is converted into a textual description using a first modality of the multimodal AI model that operates on image data, and then a second modality of the multimodal AI model which operates on text data uses the textual description to identify a change to the architecture.
Software architecture diagrams are unique, because they often include heterogeneous software architecture representations including image-based architecture diagrams and text-based architecture descriptions. A human mind cannot practically perform multimodal machine-learning operations across different data modalities in the manner recited by the claim.
The claim recites a narrowly tailored processing technique for modifying software architecture diagrams using a multimodal artificial intelligence. In particular, the claim recites converting an image of a software architecture diagram into a description using a first modality of the multimodal AI model that operates on image data, identifying a modification using a second modality of the multimodal AI model that operates on text data, and modifying the software architecture diagram based on the textual description of the modification. Thus, the claim recites a specific cross-modal processing workflow for transforming software architecture diagrams, rather than a generalized concept capable of performance in the human mind.
Thus, Applicant contends that the inquiry into the abstraction of Applicant's claims should end at this first prong as Applicant's claims do not fall into any of the enumerated groupings and therefore cannot be abstract.
Examiner’s response:
Examiner respectfully disagrees. Examiner agrees that the human mind can not operate a multimodal AI model to generate text descriptions, identify modifications, and modify a software architecture diagram. However, the human mind is capable of generating a text description given an image of a software diagram, identifying possible modifications for the software diagram, and modifying the software diagram with the aid of pen and paper. Use of a multimodal AI model, even different modes of a model, is not considered under Prong 1 as part of the mental process, but is rather considered under Prong 2 as merely applying a generic computer component to the judicial exception. Therefore, the claims recite a mental process with is completed with generic computer components, and the rejection under 35 U.S.C. 101 is proper and maintained.
In the remarks, Applicant argues:
However, for the sake of argument, even if Applicant's claims did include an abstract idea, under the second prong (2A2) of the 2019 Guidance, as asserted by the Examiner in the Office Action, the Examiner is required to show that Applicant's claims are directed to, and do not just include, an abstract idea.
"A claim is not 'directed to' a judicial exception, and thus is patent eligible, if the claim as a whole integrates the recited judicial exception into a practical application of that exception. A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception." Page 13 of the 2019 Guidance.
Applicant notes that Applicant's claims impose meaningful, practical, and succinct operations that execute multimodal AI processing on software architecture diagrams to automatically generate a modified software architecture diagram. The claim is therefore tied to a particular technological environment involving software architecture representations, graphical architecture diagrams, image-data processing, text-data processing, and modification of architecture diagrams associated with operation of computing platforms. The claim is not directed to merely displaying information, but instead recites generating a modified graphical architecture representation based on multimodal processing of image and text data.
The claim is also an improvement to the process for modifying software architecture diagrams, which are typically performed manually. The claimed process does not merely generate and present information, but instead applies multimodal AI processing to generate a modified software architecture diagram from an input software architecture diagram. This is a process which includes image-data processing and text-data processing in a dual modality architecture for automated modification of graphical architecture diagram.
In the recent decision of Ex parte Desjardins, the Appeals Review Panel (ARP) overturned a rejection under Section 101. In that case, Claim 1 included the limitation of "adjust the first values of the plurality of parameters to optimize performance of the machine learning model on the second machine learning task while protecting performance of the machine learning model on the first machine learning task." In overturning the rejection, the ARP pointed to the Federal Circuit's decision in Enfish, which observes that many advancements in computer technology, "by their very nature, may not be defined by particular physical features but rather by logical structures and processes." Enfish, LLC V. Microsoft Corp., 822 F.3d 1327, 1339 (Fed. Cir. 2016).
In the present application, Claim 1 recites use of a "first modality" of a multimodal AI model which operates on image data and which converts an image into a description of a software architecture, and a "second modality" of the multimodal AI model to identify a change to be made to the software architecture based on the description. Therefore, just as in the case of Ex parte Desjardins, the first modality optimizes performances of the multimodal AI model for the second machine learning task (second modality of the multimodal AI model) by converting the input diagram into descriptive text. The process is performed using the dual modality of the AI model which simplifies execution and optimizes the ability of the second modality to identify the change from the software architecture diagram which is now in text form, rather than image form.
Thus, Applicant's numerous claim limitations would clearly integrate an alleged abstract idea into a practical application that does not monopolize a judicial exception and are thereby patent eligible because the practical application of Applicant's claims allow for a real-world benefit through computing systems.
While applicant submits that the claimed invention is not directed to an abstract idea as discussed above, should the Office nonetheless maintain its position that the claims are directed to an abstract idea, Applicant respectfully submits that under the second step (2B) of Alice the ordered combination of elements in the independent claims are sufficient to ensure that the claim amounts to significantly more than the judicial exception.
Therefore, the claims recite elements which integrate the claims into a practical application and thus the claims recite eligible subject matter under Section 101.
Examiner’s response:
Examiner respectfully disagrees. Applicant’s use of a multimodal AI model merely recites a generic computer component which is applied to a mental step (describing a software architecture diagram, identifying modification on the software architecture diagram, and modifying the software architecture diagram). The claims do not recite a specific model, but rather applies a generic model to the steps of the mental process. In regard to Applicant’s arguments citing Desjardins and Enfish, Desjardins and Enfish recite specific improvements on a neural network model and a data structure, respectively. In contrast, the current claims do not recite a specific improvement on the multimodal AI model, but merely apply the generic multimodal AI model to a task. Therefore, the ruling of Desjardins and Enfish do not apply. Therefore, the multimodal AI model is a generic computer component merely applied to the abstract idea and does not integrate the abstract idea into practical application, nor to significantly more, and the rejection under 35 U.S.C. 101 is maintained.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VIVIAN WEIJIA DUAN whose telephone number is (703)756-5442. The examiner can normally be reached Monday-Friday 8:30AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wei Y Mui can be reached at (571) 272-3708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.W.D./Examiner, Art Unit 2191
/QING CHEN/Primary Examiner, Art Unit 2191
Prosecution Insights