CTFR 17/849,480 CTFR 98163 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. 12-151 AIA 26-51 12-51 Status of Claims This action is in reply to Applicant’s communication filed on April 14, 2025. Claims 1, 4-8, 10-12, 14-18, and 20-22 have been amended and are hereby entered. 12-151-10 AIA 12-51-10 Claim 13 has been canceled. Claims 1-8, 10-12, 14-18 and 20-22 are currently pending and have been examined. Claim Rejections - 35 USC § 112(a) 07-30-01 AIA The following is a quotation of the first paragraph of 35 U.S.C. 112(a): (a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention. The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112: The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention. 07-31-01 Claims 1-8, 10-12, 14-18 and 20-22 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Amended claims 1, 8 and 11 recite “ a plurality of medical data items ” in various lines. Examiner cannot locate anywhere in Applicant’s specification that describes what is meant by “medical data items”. For example, it’s unclear whether the “data items” entail a specific data structure such as text in a file, an entire image, annotations within an image, etc. Examiner suggests amending the claims to be consistent with the specification and/or clarifying what is meant by this amended limitation to avoid future new matter rejections under 112(a). For purposes of examination and compact prosecution, Examiner is interpreting “a plurality of medical data items” to be any sort of individual pieces of medical image-related data (Examiner further bases this off of the amendment to claim 8 which recites “ the plurality of medical data items comprise a plurality of medical image data items ”). However, clarification and correction are required by Applicant. Claims 2-7, 10, 12, 14-18 and 20-22 are dependent on claims 1 or 11 and thus are also rejected. Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-8, 10-12, 14-18 and 20-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Step 1 analysis: Claims 1 and 11 are directed to a method and a system respectively and therefore all claims fall into one of the four statutory categories. (Step 1: Yes, the claims fall into one of the four statutory categories). Step 2A analysis - Prong one: The substantially similar independent method and system claims, taking claim 1 as exemplary, recite the following limitations: A method for generating and managing annotated medical data using a computing device including a hardware processor and a memory in communication with one or more client devices and one or more databases over a network , the method comprising: retrieving medical data from a first database using the computing device including the hardware processor and the memory , wherein the medical data are retrieved in response to a query generated by a first client device , and wherein the medical data comprise a plurality of medical data items; transmitting, over the network , the medical data comprising the plurality of medical data items to the first client device for annotation; randomly selecting , by the computing device, based on a sample size, and from the plurality of medical data items, a subset of the plurality of medical data items ; transmitting, over the network , the subset of the plurality of medical data items to a second client device for annotation; receiving, by the computing device , primary annotated data from the first client device over the network , wherein the primary annotated data comprise annotations of the plurality of medical data items generated at the first client device ; receiving, by the computing device , secondary annotated data from the second client device over the network , wherein the secondary annotated data are annotations of the subset of the plurality of medical data items generated at the second client device ; comparing the primary annotated data with the secondary annotated data based on a verification condition using the computing device , wherein the verification condition includes a threshold related to an inter-observer agreement between different sets of annotated data for quality assurance of the primary annotated data ; attaching verified status metadata to the primary annotated data when the comparison of the primary annotated data with the secondary annotated data satisfies the verification condition; attaching unverified status metadata to the primary annotated data when the comparison of the primary annotated data with the secondary annotated data does not satisfy the verification condition ; generating and attaching to the primary annotated data, with the computing device , compensation share metadata indicating a respective compensation share attributable to at least one of the primary annotated data and the secondary annotated data; when the verified status metadata is attached to the primary annotated data, transmitting, over the network , the primary annotated data to a second database for storage , wherein the primary annotated data is accessible, from the second database , as training data for a machine learning model ; and when the unverified status metadata is attached to the primary annotated data, transmitting, over the network , the primary annotated data for storage, wherein the primary annotated data is accessible for adjudication. The examiner is interpreting the above bolded limitations as additional elements as further discussed below. The remaining un-bolded limitations above, as drafted, is a process that, under the broadest reasonable interpretation, covers certain methods of organizing human activity (i.e., managing personal behavior including following rules or instructions) but for recitation of generic computer components. That is, other than reciting a system implemented by a computing device (computer), the claimed invention amounts to managing personal behavior or interaction between people. For example, but for the bolded limitations above, this claim encompasses a first and a second person annotating the same medical data and comparing the two annotations to a condition and then based on the comparison, assigning a status to the annotated medical data in the manner described in the identified abstract idea, supra. The Examiner notes that certain “method[s] of organizing human activity” includes a person’s interaction with a computer (see MPEP 2106.04(a)(2)(II)). Examiner further notes that the limitations regarding the “transmitting” of data is a consequence of the claim being confined to a computer (i.e., a computerized method) and is therefore being interpreted as part of the abstract idea and not as an additional element. If a claim limitation, under its broadest reasonable interpretation, covers managing personal behavior or interactions between people but for the recitation of generic computer components, then it falls within the “certain methods of organizing human activity” grouping of abstract ideas. Accordingly, the claim recites an abstract idea. (Step 2A – Prong 1: Yes, the claims are abstract). Step 2A analysis - Prong two: Claims 1 and 11 recite additional elements beyond the abstract idea. Claim 1 recites a computing device including a hardware processor and a memory in communication with one or more client devices (a first and a second client device) and one or more databases (a first and second database) over a network and a machine learning model. Claim 11 recites a first and a second database, a network, a first and second client device implemented with a hardware processor and a memory, a user interface, a computing device including a hardware processor and a memory, and a machine learning model. The claims are applying generic computer components to the recited abstract limitations. This judicial exception is not integrated into a practical application. In particular, the claims recite a first and a second database, a network, a first and second client device, a user interface, a computing device including a hardware processor and a memory, and a machine learning model which are recited at a high-level of generality (i.e., as a generic processor performing generic computer functions) such that it amounts to no more than mere instructions to apply the exceptions using a generic computer component. For example, Applicant’s specification explains that the computer system stores instructions, implements instructions, receives electrical inputs, outputs via an interface, etc. (see Applicant’s specification pages 16, 19-20). Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, Claims 1 and 11 are directed to an abstract idea without practical application. ( Step 2A – Prong 2: No, the additional claimed elements are not integrated into a practical application). Step 2B analysis: For the next step of the analysis, it must be determined whether the limitations present in the claims represent a patent-eligible application of the abstract idea. A claim directed to a judicial exception must be analyzed to determine whether the elements of the claim, considered both individually and as an ordered combination are sufficient to ensure that the claim as a whole amounts to significantly more than the exception itself. For the role of a computer in a computer implemented invention to be deemed meaningful in the context of this analysis, it must involve more than performance of well-understood, routine, and conventional activities previously known to the industry. Further, the mere recitation of a generic computer cannot transform a patent ineligible abstract idea into a patent-eligible invention. See MPEP 2106.05(d). Applicant’s specification discloses the following: Applicant describes embodiments of the disclosure at a very high level to include the use of a wide variety of processors, memories, databases, storage devices, software, computers, etc. (see Applicant’s Spec. pages 16, 19-20). Generic computer components recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system. Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. The collective functions appear to be implemented using conventional computer systemization. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of a first and a second database, a network, a first and second client device, a user interface, a computing device and a machine learning model to perform all of the steps discussed above amount to no more than mere instructions to apply the exceptions using generic computer components. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claims do not provide an inventive concept significantly more than the abstract idea. Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea recited in claims 1 and 11 into a practical application because they do not impose any meaningful limits on practicing the abstract idea. (Step 2B: No, the claims do not provide significantly more). Dependent Claims 2-8, 10, 12, 14-18 and 20-22 further define the abstract idea that is presented in independent Claims 1 and 11, and are further grouped as certain methods of organizing human activity and are abstract for the same reasons and basis as presented above. Further, Claims 2-5, 12, 14 and 16-18 recite additional elements beyond the abstract idea. Claims 2-3 and 12 recite a compensation model, which appears to be software. Claims 4-5, 14, and 16-18 recite a third client device. Claim 12 further recites a memory. These additional elements are recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component. For example, as noted above, the Applicant’s specification indicates the use of known processors, memories and client devices. Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claims do not recite additional elements that integrate the judicial exception into a practical application when considered both individually and as an ordered combination. Therefore, the dependent claims are also directed to an abstract idea. Thus, Claims 1-8, 10-12, 14-18 and 20-22 are rejected under 35 U.S.C. 101 as being directed to abstract ideas without significantly more. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-20-02-aia AIA This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention. 07-21-aia AIA Claim s 1-8, 10-12, 14-18 and 20-22 are rejected under 35 U.S.C. 103 as being unpatentable over Abedin et al. (US 20190258663) in view of Diedrich et al. (US 20220270146), in view of Banipal et al. (US 20210042290), further in view of Ghosh et al. (US 6978458) . Regarding Claim 1 , Abedin disclose the following limitations: A method for generating and managing annotated medical data using a computing device including a hardware processor and a memory in communication with one or more client devices and one or more databases over a network, the method comprising: retrieving medical data from a first database using the computing device including the hardware processor and the memory, (Abedin discloses a collections management service platform using a computer system including a memory and one or more processors ( a computing device including a hardware processor and a memory – paras 18-21, 52) that provides collection annotations of medical data ( retrieving medical data from a first database using the computing device including the hardware processor and the memory ). Abedin discloses the use of a variety of data stores that may provide medical data to or receive data from the Collections Management Service, including a Picture Archiving and Communication System (PACS), EHR, or EMR, etc. Various databases, data banks, and data stores are used ( one or more databases ) for indexing collections and annotations. Computer system/server 12 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices ( in communication with one or more client devices ) that are linked through a communications network ( over a network ). – paras 6-7, 12-22, 37-38, 51-52; FIG. 1) wherein the medical data are retrieved in response to a query generated by a first client device , and wherein the medical data comprises a plurality of medical data items ; (Abedin discloses search ( a query ) and retrieval ( retrieved ) of medical data ( the medical data ) via a user interface ( a first client device ), including collections of images and annotations of medical data ( wherein the medical data comprises a plurality of medical data items ). – paras 12, 16, 20-21, 26) transmitting, over the network, the medical data comprising the plurality of medical data items to the first client device for annotation; (Abedin discloses receiving a plurality of medical images ( transmitting e medical data comprising the plurality of medical data items ) and providing the images to each of a plurality of remote users for annotation ( to the first client device for annotation ). – paras 3, 8, 48; FIG. 5) … selecting, by the computing device, based on a sample size, and from the plurality of medical data items, a subset of the plurality of medical data items; transmitting, over the network, the subset of the plurality of medical data items to a second client device for annotation; (Abedin discloses that a collection of medical images is formed containing a subset ( selecting a subset of the plurality of medical data items ) of the plurality of medical images received ( based on a sample size, and from the plurality of medical data items ). One or more images from a collection ( the subset of the plurality of medical data items ) is provided ( transmitting ) to each of a plurality of remote users such as at least two users ( to a second client device for annotation ). – abstract; paras 3, 48; FIG. 5 items 502-503; Claim 8) receiving, by the computing device, primary annotated data from the first client device over the network, wherein the primary annotated data comprise annotations of the plurality of medical data items generated at the first client device; (Abedin discloses providing the formed collections of medical images ( the plurality of medical data items ) to remote users and then receiving annotations ( receiving annotations of the plurality of medical data items generated at the first client device ) from each of the plurality of remote users ( primary annotated data generated at the first client device ). – abstract; paras 37-38, 46, 48; FIG. 5 item 505; Claim 9) receiving, by the computing device, secondary annotated data from the second client device over the network, wherein the secondary annotated data are annotations of the subset of the plurality of medical data items generated at the second client device; (Abedin discloses that a subset of the plurality of medical images are provided to the users and image annotations are then received ( receiving annotations of the subset of the plurality of medical data items generated at the second client device ) from each of the plurality of remote users ( secondary annotated data from the second client device ). – abstract; paras 37-38, 46, 48; FIG. 5 item 505; Claim 9) comparing the primary annotated data with the secondary annotated data based on a verification condition using the computing device, (Abedin discloses that the data model for a collection can handle multiple annotators across tasks, and multiple annotations for the same attribute by different annotators for cross-validation ( comparing the primary annotated data with the secondary annotated data based on a verification condition ). – paras 37) Abedin does not disclose the following limitations met by Diedrich: wherein the verification condition includes a threshold related to an inter-observer agreement between different sets of annotated data for quality assurance of the primary annotated data ; (Diedrich teaches inter-rater reliability which may include the ability to compare peer to peer quality for consistency to ensure that images are being interpreted at a higher quality ( for quality assurance of the primary annotated data ), a higher quality as ranked by the annotator ( an inter-observer agreement between different sets of annotated data ). The image quality threshold or the threshold ( the verification condition includes a threshold ) may be set based on set parameters, such as a percentage of the inter-rater reliability ( related to an inter-observer agreement between different sets of annotated data ). If the threshold is met, the annotation data or the annotation ownership data may be recorded in the blockchain ledger. Measuring the quality of the input data, such as the image annotation quality, may allow a ranking of the annotators in the marketplace such that the higher quality training data can be rewarded. – paras 22, 49-51; FIG. 2, item 208; Claim 5) attaching verified status metadata to the primary annotated data when the comparison of the primary annotated data with the secondary annotated data satisfies the verification condition; attaching unverified status metadata to the primary annotated data when the comparison of the primary annotated data with the secondary annotated data does not satisfy the verification condition ; (Diedrich teaches that if the annotation quality threshold has been met ( satisfies the verification condition ), then the annotation quality metrics are provided/stored to the marketplace ( attaching verified status ). If the threshold is not met ( does not satisfy the verification condition ), then the image may not be stored to the marketplace as a quality image ( attaching unverified status ). Inter-rater reliability may include the ability to compare peer to peer quality for consistency to ensure that images are being interpreted at a higher quality ( when the comparison of the primary annotated data with the secondary annotated data ). Further, Diedrich teaches that the quality measures may be published in multiple ways such as stored in a private tag in DICOM or published in the metadata ( attaching verified/unverified status metadata to the primary annotated data – para 57) because, e.g., higher market prices are determined for higher quality annotation data. – paras 49, 54-57; FIG. 2, items 208-212) generating and attaching to the primary annotated data, with the computing device , compensation share metadata indicating a respective compensation share attributable to at least one of the primary annotated data and the secondary annotated data (Diedrich teaches the use of smart contracts with the blockchain network to tie or connect the set of images and annotations to annotators. The annotators ( the primary annotated data and the secondary annotated data ) are ranked and given a quality score such that a higher quality training data can be rewarded with higher prices ( compensation share indicating a respective compensation share ). The quality scores may be provided as background data or as metadata ( generating and attaching to the primary annotated data – para 22). – paras 20-22, 42-43, 49) when the verified status metadata is attached to the primary annotated data, transmitting, over the network, the primary annotated data to a second database for storage, wherein the primary annotated data is accessible, from the second database, as training data for a machine learning model; (Diedrich teaches that the annotations that have met the quality threshold or the quality thresholds ( when the verified status metadata is attached to the primary annotated data ) may be saved and encrypted in the cloud object storage server ( transmitting, over the network, the primary annotated data to a second database for storage, ) and the annotations associated with the image may be referenced in the blockchain ledger. The annotation quality metrics are provided to marketplace and may be accessed ( the primary annotated data is accessible ) to train the machine learning model ( as training data for a machine learning model ). – paras 54, 58-60; FIG. 2, items 208 and 212-216) While Abedin discloses a payment process where the number of annotations done by each user is recorded and compensation is provided through a payment system at a rate reflecting the total annotations performed ( generating compensation share metadata indicating a respective compensation share attributable at least one of the primary annotator and the secondary annotator – see Abedin para 32; Claim 15), Abedin does not disclose attaching compensation share metadata to the annotated data . However, Diedrich teaches the limitations not met by Abedin, as noted above. It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified receiving and storing the annotations from a plurality of users as disclosed by Abedin to incorporate comparing annotations to a quality threshold and publishing the corresponding metadata as taught by Diedrich and to have further modified the payment system as disclosed by Abedin to incorporate providing, in the form of metadata, quality scores related to pricing as taught by Diedrich in order to improve the technical field of machine learning by creating a model that annotates images and improves the quality of the annotations (see Diedrich para 15). Abedin and Diedrich do not teach the following limitations met by Banipal: and when the unverified status metadata is attached to the primary annotated data, transmitting, over the network, the primary annotated data for storage, wherein the primary annotated data is accessible for adjudication. (Banipal teaches that as a method of quality control, an overlap of review is utilized, wherein two or more annotators may review the same document and attach different annotations to the same element ( when the unverified status metadata is attached to the primary annotated data ). It is the function of an adjudicator to review the conflicting annotations and make a determination of which annotation is correct ( the primary annotated data is accessible for adjudication ). – para 26) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have further modified receiving and storing annotations from a plurality of users as disclosed by Abedin to incorporate adjudicator review and dynamically adjusting metadata as taught by Banipal in order to increase efficiency of the adjudication process (see Banipal para 26). Abedin, Diedrich and Banipal do not disclose the following limitations met by Ghosh: - randomly selecting… a subset of the plurality of medical data items; (Ghosh teaches techniques for distributing data items of a particular set of data to a plurality of buckets. Subsets of a data set are formed by randomly selecting ( randomly selecting ) data items. For example, for each subset of the set of data, a sample of e.g., 50 data items are randomly selected ( randomly selecting a subset of the plurality of medical data items ). – col 2, lines 56-63; col 4, lines 53-56; col 4, lines 65-67) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have further modified forming a subset of the plurality of medical images as disclosed by Abedin to incorporate randomly selecting data items for each subset of the set of data as taught by Ghosh in order to decrease the time it takes for a computer system to perform a task (see Ghosh col 1, lines 14-17). Regarding Claim 2, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: wherein the compensation share metadata is generated according to a compensation model. (Abedin discloses that compensation is provided through a payment system ( a compensation model ). – para 32) Regarding Claim 3, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: wherein the compensation model indicates a first compensation share to the primary annotated data and a second compensation share to the secondary annotated data, (Abedin discloses that the annotators are compensated at a rate reflecting the total annotations performed ( a first/second compensation share to the primary/secondary annotated data ), indicating that each annotator is paid a specific amount according to the payment system. – para 32) wherein the first compensation share is greater than the second compensation share. (Abedin discloses that each annotator may be compensated an amount reflecting the total annotations performed. This indicates that if one user performs more annotations than another user, they will be compensated accordingly ( the first compensation share is greater than the second ). – para 32) Regarding Claim 4, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: receiving, by the computing device , third annotated data from the third client device over the network, wherein the third annotated data comprise medical data annotations generated at the third client device; (Abedin discloses receiving image annotations ( medical data annotations ) from each of the plurality of remote users ( third annotated data comprise medical data annotations generated at the third client device ). – abstract; paras 37-38, 46, 48; FIG. 5 item 505; Claim 9) transmitting, over the network, annotated data having attached thereto unverified status metadata to a third client device; (Banipal teaches that, as a method of quality control, an overlap of review is utilized, wherein two or more annotators may review the same document and attach different annotations to the same element. In this occurrence, it is the function of an adjudicator to review the conflicting annotations and make a determination of which annotation is correct. – paras 26) and updating the compensation share metadata in response to the third annotated data. (Banipal teaches that the assigned annotations are assessed for accuracy and a score associated with the metadata is adjusted ( updating the…metadata ). – abstract; paras 7, 33) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the payment system as disclosed by Abedin to incorporate adjudicator review and dynamically adjusting metadata as taught by Banipal in order to increase efficiency of the adjudication process (see Banipal para 26). Regarding Claim 5, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: wherein the compensation share metadata are updated to allocate a first compensation share to the primary annotated data and a second compensation share to the third annotated data when the third annotated data align with the primary annotated data . (Diedrich teaches that each annotator is compensated ( allocate a first compensation share to the primary annotated data and a second compensation share to the third annotated data ) at a rate reflecting the quality of their annotations performed ( the compensation share metadata ). Higher quality results in higher reward and lower quality results in lower reward. Quality of the images may be determined based on the quality threshold set by the inter rater reliability being at a predetermined percentage ( when the third annotated data align with the primary annotated data ). – paras 21-22, 49, 51-52) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the payment system as disclosed by Abedin to incorporate compensation rates reflecting annotation quality as taught by Diedrich in order to improve the technical field of machine learning by creating a model that annotates images and improves the quality of the annotations (see Diedrich para 15). Regarding Claim 6, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: wherein the compensation share metadata are updated to allocate a first compensation share to the secondary annotated data and a second compensation share to the third annotated data when the third annotated data align with the secondary annotated data . (Diedrich teaches that each annotator is compensated ( allocate a first compensation share to the secondary annotated data and a second compensation share to the third annotated data ) at a rate reflecting the quality of their annotations performed ( the compensation share metadata ). Higher quality results in higher reward and lower quality results in lower reward. Quality of the images may be determined based on the quality threshold set by the inter rater reliability being at a predetermined percentage ( when the third annotated data align with the primary annotated data ). – paras 21-22, 49, 51-52) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the payment system as disclosed by Abedin to incorporate compensation rates reflecting annotation quality as taught by Diedrich in order to improve the technical field of machine learning by creating a model that annotates images and improves the quality of the annotations (see Diedrich para 15). Regarding Claim 7, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: wherein the compensation share metadata are updated to allocate a first compensation share to the primary annotated data, a second compensation share to the secondary annotated data, and a third compensation share to the third annotated data when both the primary annotated data and the secondary annotated data failed to align with the third annotation data . (Diedrich teaches that each annotator is compensated ( allocate a first compensation share to the primary annotated data, a second compensation share to the secondary annotated data, and a third compensation share to the third annotated data ) at a rate reflecting the quality of their annotations performed ( the compensation share metadata ). Higher quality results in higher reward and lower quality results in lower reward. Quality of the images may be determined based on the quality threshold set by the inter rater reliability being at a predetermined percentage ( when both the primary annotated data and the secondary annotated data failed to align with the third annotation data ). – paras 21-22, 49, 51-52) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the payment system as disclosed by Abedin to incorporate compensation rates reflecting annotation quality as taught by Diedrich in order to improve the technical field of machine learning by creating a model that annotates images and improves the quality of the annotations (see Diedrich para 15). Regarding Claim 8, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: wherein the plurality of medical data items comprise a plurality of medical image data items , and wherein the primary annotated data comprise at least one of labeled medical image data or segmented medical image data. (Abedin discloses that the data may be medical images and labeled images ( labeled medical image data ) for disease occurrences. – abstract; paras 11, 19-21, 26, 34, 39) Regarding Claim 10, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: wherein the compensation share metadata are attached to the plurality of medical image data items as an exam series level of a DICOM header in each image of the plurality of medical image data items . (Abedin discloses using DICOM standards and that DICOM provides a standard for handling, storing, printing, and transmitting information in medical imaging. – paras 13, 46-47) Regarding Claim 11, Abedin discloses the following limitations: A medical data annotation management system, comprising: a first database storing medical data comprising a plurality of medical data items ; (Abedin discloses using various databases, data banks, and data stores ( a first database ) for indexing collections and annotations of images ( storing medical data comprising a plurality of medical data items ). – paras 13-15, 20-21; FIG. 1) a first client device implemented with a hardware processor and a memory, the first client device being configured to generate a user interface for annotating the plurality of medical data items received from the first database, and to generate first annotated medical data comprising annotations of the plurality of medical data items ; (Abedin discloses a user interface ( the first client device being configured to generate a user interface – FIGs. 2-3) for annotation of large image datasets. A plurality of images are received from a plurality of data stores and provided to a plurality of remote users for annotation ( annotating the plurality of medical data items received from the first database - paras 20, 48). Computer system/server 12 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices ( a first client device implemented with a hardware processor and a memory – para 51) that are linked through a communications network. And the annotations are received from each of the plurality of remote users ( generate first annotated medical data comprising annotations of the plurality of medical data items – para 48). – paras 20, 26, 33-34, 48-52; FIGs. 2-3, 5-6) a second client device implemented with a hardware processor and a memory, the second client device being configured to generate a user interface for annotating a subset of the plurality of medical data items received from the first database, and to generate second annotated medical data , wherein the second annotated medical data are annotations of the subset of the plurality of medical data items ; (Abedin discloses a user interface ( configured to generate a user interface – FIGs. 2-3) for annotation of large image datasets. A subset of the plurality of medical images are provided to the users and image annotations are then received ( for annotating a subset of the plurality of medical data items ) from each of the plurality of remote users ( generate second annotated medical data , wherein the second annotated medical data are annotations of the subset of the plurality of medical data items ). Computer system/server 12 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices ( a second client device implemented with a hardware processor and a memory – para 51) that are linked through a communications network. A plurality of images are received from a plurality of data stores and provided to a plurality of remote users for annotation ( annotating medical data received from the first database - paras 20, 48). – abstract; paras 20, 37-38, 46, 48, 51; FIG. 5 items 502, 505; Claim 9) a computing device implemented with a hardware processor and a memory and in communication with the first database, the first client device , and the second client device , over a network, the computing device being configured to: (Abedin discloses a Collections Management Service and a computing node ( a computing device implemented with a hardware processor and a memory ) capable of performing any of the functionality set forth hereinabove. Computer system/server 12 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network ( in communication with the first database, the first client device , and the second client device ). – paras 12, 35, 51) generate a work order to retrieve the plurality of medical data items from the first database; (Abedin discloses that the variety of data stores may provide medical data ( the plurality of medical data items from the first database ) to the Collections Management Service through a search and retrieval process ( a work order to retrieve ). – paras 12-16, 20-21, 26) transmit, over the network, the plurality of medical data items to the first client device ; (Abedin discloses providing one or more images ( the plurality of medical data items ) to each of a plurality of remote users via remote processing devices that are linked through a communications network ( transmit, over the network, to the first client device ). – paras 26, 48, 51; FIG. 5, item 503) …select, based on a sample size and from the plurality of medical data items, one or more medical data items to be the subset of the plurality of medical data items; transmit the subset of the plurality of medical data items to the second client device , over the network; (Abedin discloses that a collection of medical images is formed containing a subset ( select one or more medical data items to be the subset of the plurality of medical data items ) of the plurality of medical images received ( based on a sample size and from the plurality of medical data items ). One or more images from a collection ( the subset of the plurality of medical data items ) is provided ( transmitting ) to each of a plurality of remote users such as at least two users ( to a second client device for annotation ). – abstract; paras 3, 48; FIG. 5 items 502-503; Claim 8) receive, over the network, the first annotated medical data from the first client device ; (Abedin discloses receiving image annotations from each of the plurality of remote users ( first annotated data from the first client device ). – abstract; paras 37-38, 46, 48; FIG. 5 item 505; Claim 9) Receive, over the network, the second annotated medical data from the second client device ; (Abedin discloses receiving image annotations from each of the plurality of remote users ( second annotated data from the first client device ). – abstract; paras 37-38, 46, 48; FIG. 5 item 505; Claim 9) compare the first annotated medical data with the second annotated medical data, based on a verification condition,…; (Abedin discloses that the data model for a collection can handle multiple annotators across tasks, and multiple annotations for the same attribute by different annotators for cross-validation ( compare the first annotated medical data with the second annotated medical data, based on a verification condition ). – para 37) wherein the compensation share metadata indicates: a first entity of the first client device a first compensation share associated with the first annotated medical data, a second entity of the second client device and a second compensation share associated with the second annotated medical data; (Abedin discloses a payment process where the number of annotations done by each user is recorded and compensation is provided through a payment system at a rate reflecting the total annotations performed ( a first entity of the first client device a first compensation share associated with the first annotated medical data, a second entity of the second client device and a second compensation share associated with the second annotated medical data ). – paras 32, 34; Claim 15) Abedin does not disclose the following limitations met by Diedrich: wherein the verification condition includes a threshold related to an inter-observer agreement between different sets of annotated data for quality assurance of the first annotated medical data; (Diedrich teaches inter-rater reliability which may include the ability to compare peer to peer quality for consistency to ensure that images are being interpreted at a higher quality ( for quality assurance of the primary annotated data ), a higher quality as ranked by the annotator ( an inter-observer agreement between different sets of annotated data ). The image quality threshold ( the verification condition includes a threshold ) may be set based on set parameters, such as a percentage of the inter-rater reliability ( related to an inter-observer agreement between different sets of annotated data ). If the threshold is met, the annotation data or the annotation ownership data may be recorded in the blockchain ledger. Measuring the quality of the input data, such as the image annotation quality, may allow a ranking of the annotators in the marketplace such that the higher quality training data can be rewarded. – paras 22, 49-51; FIG. 2, item 208; Claim 5) attaching verified status metadata to the first annotated medical data when the comparison of the first annotated medical data with the second annotated medical data satisfies the verification condition; attaching unverified status metadata to the first annotated medical data when the comparison of the first annotated medical data with the second annotated medical data does not satisfy the verification condition; (Diedrich teaches that if the annotation quality threshold has been met ( satisfies the verification condition ), then the annotation quality metrics are provided/stored to the marketplace ( attaching verified status ). If the threshold is not met ( does not satisfy the verification condition ), then the image may not be stored to the marketplace as a quality image ( attaching unverified status ). Inter-rater reliability may include the ability to compare peer to peer quality for consistency to ensure that images are being interpreted at a higher quality ( when the comparison of the first annotated medical data with the second annotated medical data ). Further, Diedrich teaches that the quality measures may be published in multiple ways such as stored in a private tag in DICOM or published in the metadata ( attaching verified/unverified status metadata to the first annotated medical data – para 57) because, e.g., higher market prices are determined for higher quality annotation data. – paras 49, 54-57; FIG. 2, items 208-212) index compensation share metadata to the first annotated data,… (Diedrich teaches the use of smart contracts with the blockchain network to tie or connect the set of images and annotations to annotators. The annotators are ranked and given a quality score such that a higher quality training data can be rewarded with higher prices ( compensation share metadata ). Inter-rater reliability includes the ability to compare peer to peer quality for consistency to ensure that images are being interpreted at a higher quality, a higher quality as ranked by the annotator. The quality scores may be provided as background data or as metadata ( index compensation share metadata to the first annotated data – para 22). – paras 20-22, 42-43, 49-51) when the verified status metadata is attached to the first annotated data, transmit, over the network, the first annotated medical data to a second database for storage, wherein the first annotated medical data is accessible, from the second database, as training data for a machine learning model; (Diedrich teaches that the annotations that have met the quality threshold or the quality thresholds ( when the verified status metadata is attached to the first annotated data ) may be saved and encrypted in the cloud object storage server ( transmit, over the network, the first annotated data to a second database for storage, ) and the annotations associated with the image may be referenced in the blockchain ledger. The annotation quality metrics are provided to marketplace and may be accessed ( the first annotated data is accessible ) to train the machine learning model ( as training data for a machine learning model ). – paras 54, 58-60; FIG. 2, items 208 and 212-216) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified receiving and storing the annotations from a plurality of users as disclosed by Abedin to incorporate comparing annotations peer to peer for quality purposes and publishing the corresponding metadata as taught by Diedrich and to have further modified the payment system as disclosed by Abedin to incorporate providing, in the form of metadata, quality scores related to pricing as taught by Diedrich in order to improve the technical field of machine learning by creating a model that annotates images and improves the quality of the annotations (see Diedrich para 15). Abedin and Diedrich do not teach the following limitations met by Banipal: and when the unverified status metadata is attached to the first annotated medical data, transmit, over the network, the first annotated medical data for storage, wherein the first annotated medical data is accessible for adjudication. (Banipal teaches that as a method of quality control, an overlap of review is utilized, wherein two or more annotators may review the same document and attach different annotations to the same element ( when the unverified status metadata is attached to the first annotated medical data ). It is the function of an adjudicator to review the conflicting annotations and make a determination of which annotation is correct ( the first annotated medical data is accessible for adjudication ). – para 26) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have further modified receiving and storing annotations from a plurality of users as disclosed by Abedin to incorporate adjudicator review and dynamically adjusting metadata as taught by Banipal in order to increase efficiency of the adjudication process (see Banipal para 26). Abedin, Diedrich and Banipal do not disclose the following limitations met by Ghosh: randomly select…one or more medical data items to be the subset of the plurality of medical data items; (Ghosh teaches techniques for distributing data items of a particular set of data to a plurality of buckets. Subsets of a data set are formed by randomly selecting ( randomly select ) data items. For example, for each subset of the set of data, a sample of e.g., 50 data items are randomly selected ( randomly select one or more medical data items to be the subset of the plurality of medical data items ). – col 2, lines 56-63; col 4, lines 53-56; col 4, lines 65-67) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have further modified forming a subset of the plurality of medical images as disclosed by Abedin to incorporate randomly selecting data items for each subset of the set of data as taught by Ghosh in order to decrease the time it takes for a computer system to perform a task (see Ghosh col 1, lines 14-17). Regarding Claim 12, this claim depends on claim 11, which is rejected for the basis and reasons disclosed above. Additionally, it recites substantially similar limitations to those recited in claims 2-3 above; thus, the same rejection applies. Regarding Claim 14, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: further comprising a third client device implemented with a hardware processor and a memory, the third client device being configured to generate a user interface for adjudicating the first annotated medical data, and to generate adjudicated annotated data; (Banipal teaches that, as a method of quality control, an overlap of review is utilized, wherein two or more annotators may review the same document and attach different annotations to the same element. In this occurrence, it is the function of an adjudicator to review the conflicting annotations and make a determination of which annotation is correct ( generate a user interface for adjudicating the first annotated medical data, and to generate adjudicated annotated data ). – paras 26) wherein the computing device is configured to index updated metadata to the adjudicated annotated data. (Banipal teaches that the assigned annotations are assessed for accuracy and a score associated with the metadata is adjusted ( index updated metadata to the adjudicated annotated data ). – abstract; paras 7, 33) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have further modified the payment system as disclosed by Abedin to incorporate adjudicator review and dynamically adjusting metadata as taught by Banipal in order to increase efficiency of the adjudication process (see Banipal para 26). Regarding Claim 15, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: wherein the computing device is configured to calculate a third compensation share based on the adjudicated annotated data. (Abedin discloses that the plurality of remote users (i.e., annotators) are compensated ( calculate a third compensation share ) at a rate reflecting the total annotations performed ( based on the adjudicated annotated data ), indicating that each annotator is paid a specific amount according to the payment system. – para 32) Regarding Claim 16, this claim depends on claim 14, which is rejected for the basis and reasons disclosed above. Additionally, it recites substantially similar limitations to those recited in claim 5 above; thus, the same rejection applies. Regarding Claim 17, this claim depends on claim 14, which is rejected for the basis and reasons disclosed above. Additionally, it recites substantially similar limitations to those recited in claim 6 above; thus, the same rejection applies. Regarding Claim 18, this claim depends on claim 14, which is rejected for the basis and reasons disclosed above. Additionally, it recites substantially similar limitations to those recited in claim 7 above; thus, the same rejection applies. Regarding Claim 20, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: The method of claim 1, further comprising: detecting a retrieval of the primary annotated data having the verified status metadata attached; (Diedrich teaches that the annotations that have met the quality threshold or the quality thresholds may be saved and encrypted in the cloud object storage server ( detecting a retrieval of the primary annotated data having the verified status metadata attached ) and the annotations associated with the image may be referenced in the blockchain ledger. – paras 55, 58) extracting, based on the compensation share metadata attached to the primary annotated data having the verified status metadata attached, the respective compensation share attributable to at least one of the primary annotated data and the secondary annotated data in relation to the retrieval of the annotated data having the verified status metadata attached; (Diedrich teaches that the annotations that have met the quality threshold(s) may be saved and encrypted in the cloud object storage server and the annotations associated with the image may be referenced in the blockchain ledger ( in relation to the retrieval of the annotated data having the verified status metadata attached ). The quality measures may be published in the metadata and determine the market prices. The model trainers may view if the annotations being purchased are from a higher purchase price from high ranked experts or from lower purchase prices from lower ranked users or crowd sourced users ( extracting the respective compensation share attributable to at least one of the primary annotated data and the secondary annotated data ). – paras 57-58, 66) and controlling allocation of the respective compensation share to at least one entity associated with at least one of the primary annotated data and the secondary annotated data. (Diedrich teaches that if an annotator or a model trainer would like to make a purchase using the marketplace, the transactions are recorded in a private blockchain ledger and Smart contacts may be used to execute the transactions ( controlling allocation of the respective compensation share ). Annotators and model trainers may view the annotations before purchasing them using a zero client viewer ( associated with at least one of the primary annotated data and the secondary annotated data ). – paras 70-71) It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified receiving and storing the annotations from a plurality of users as disclosed by Abedin to incorporate ranking annotators work quality, publishing quality measures in the metadata, and determining the prices based on the rankings as taught by Diedrich in order to improve the technical field of machine learning by creating a model that annotates images and improves the quality of the annotations (see Diedrich para 15). Regarding Claim 21, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: The method of claim 1, further comprising: updating at least one entity profile based on the comparison of the primary annotated data with the secondary annotated data, wherein updating the at least one entity profile includes updating at least one of: an indicator of verified annotations; an indicator of unverified annotations; a success rate; or a failure rate . (Diedrich teaches that quality scores may be given to annotators for different annotation tasks to measure the annotators strongest specialties where they produce the most valuable annotations to a marketplace for training data purposes. Quality scores, for example, may be provided as background data or as metadata and may be grouped from the rankings. Inter-rater reliability may include the ability to compare peer to peer quality for consistency to ensure that images are being interpreted at a higher quality, a higher quality as ranked by the annotator ( based on the comparison of the primary annotated data with the secondary annotated data ). The rankings may be analyzed over a period of time ( updating at least one entity profile ), for example, when an annotator is identified as being consistent with the provided annotations, when an annotator has a specific specialty or when an annotator consistently provides a similar level of annotations as the annotator's high ranking peers ( updating a success rate; or a failure rate ). – paras 49, 22). It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified receiving and storing the annotations from a plurality of users as disclosed by Abedin to incorporate ranking annotators work quality over time as taught by Diedrich in order to improve the technical field of machine learning by creating a model that annotates images and improves the quality of the annotations (see Diedrich para 15). Regarding Claim 22, Abedin, Diedrich, Banipal and Ghosh disclose every limitation above and further disclose the following limitations: The system of claim 11, wherein the computing device is configured to: determine whether to change a role designation of the first entity based on the comparison of the first annotated medical data with the second annotated medical data; (Diedrich teaches that measuring the image annotation quality, may allow a ranking of the annotators ( determine whether to change a role designation ) in the marketplace such that the higher quality training data can be rewarded. Inter-rater reliability may include the ability to compare peer to peer quality for consistency to ensure that images are being interpreted at a higher quality, a higher quality as ranked by the annotator ( based on the comparison of the first annotated medical data with the second annotated medical data ).– paras 49, 22) and in response to a determination to change the role designation of the first entity: determine an updated role designation of the first entity, and update an entity profile of the first entity to reflect the updated role designation. (Diedrich teaches that measuring the quality of the input data, such as the image annotation quality, may allow a ranking of the annotators in the marketplace ( in response to a determination to change the role designation ) such that the higher quality training data can be rewarded. The rankings may be analyzed over a period of time ( determine an updated role designation and update an entity profile to reflect the updated role designation ), for example, when an annotator is identified as being consistent with the provided annotations, when an annotator has a specific specialty or when an annotator consistently provides a similar level of annotations as the annotator's high ranking peers. – para 22). It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified receiving and storing the annotations from a plurality of users as disclosed by Abedin to incorporate ranking annotators work quality over time as taught by Diedrich in order to improve the technical field of machine learning by creating a model that annotates images and improves the quality of the annotations (see Diedrich para 15). Response to Arguments Regarding rejections under 35 USC § 101 to Claims 1-8, 10-12, 14-18 and 20-22, Applicant’s arguments have been fully considered, and are not persuasive. The rejection has been updated in light of latest amendments. Applicant argues: (a) The claim does not recite directing people to follow certain rules or instructions. Claim 1 is rooted in technology. While the recited "first client device" or "second client device" may be used by a person, the claim does not recite rules or instructions for such a person to follow. Instead, claim 1 recites using a computing device in communication with database(s) and client device(s) via a network to process annotations of medical data items for training a machine learning model. The Office cannot reasonably remove the computing system features from the analysis and then contend that the remaining features are about managing personal behavior. See Office Action, pp. 4-5. The Office's oversimplification or even mischaracterization of the claim as managing personal behavior improperly ignored the fact that the claim is rooted in technology for training a machine learning model. Annotations by a person outside the context of a computing system (e.g., writings or drawings on a piece of paper) are in a form that cannot be used to train a machine learning model directly or practically, if at all. (p. 10-11). Regarding (a), Examiner respectfully disagrees. MPEP 2106.04(a)(2)(II) states that a claimed invention is directed to certain methods of organizing human activity if the identified claim elements contain limitations that encompass fundamental economic behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions). The Examiner submits that the claims recite a series of rules or instructions that a person or persons, with or without the aid of a computer, would follow to manage annotated medical data quality. The Examiner notes that Applicants Background describes annotating medical data (see Applicant’s Spec. paras 2, 4) as a human task. Further, the Examiner notes that certain “method[s] of organizing human activity” includes a person’s interaction with a computer (see MPEP 2106.04(a)(2)(II)). Further, Examiner notes that multiple CAFC decisions that the Office has characterized as Certain Methods of Organizing Human Activity did not actively recite a person or persons performing the steps of the claims ( see e.g., EPG, TLI communications, Ultramercial ). Because whether a human is required to perform the steps of the claim is not a requirement for claims to encompass certain methods of organizing human activity, this argument is not persuasive. (b) Here, claim 1 recites features relating to the training data for machine learning models. The recited features reflect improvements in training a machine learning model. The PTAB held that such improvements integrate an alleged abstract idea into a practical application. For example, in Ex parte Desjardins , Appeal 2024-000567 (PTAB Nov. 4, 2025) (precedential), the Director held that the claims at issue reciting improvements in training a machine learning model integrate an abstract idea into a practical application and are thus patent-eligible. Much like in Desjardins and Carmody , claim 1 of the present application recites features that reflect improvements to training a machine learning model, and thus is patent-eligible. The features recited in claim 1 allow for faster annotation of a large amount of medical data and also verification of the annotation quality for training a machine learning model, as an efficient way to provide more training data with quality assurance. (p. 11-13). Regarding (b), Examiner respectfully disagrees. Claim 1 recites the following regarding training a machine learning model: when the verified status metadata is attached to the primary annotated data, transmitting, over the network, the primary annotated data to a second database for storage, wherein the primary annotated data is accessible, from the second database, as training data for a machine learning model . Claim 1 does not recite training a machine learning model, rather it recites that the data stored in the second database is “accessible as training data for a machine learning model”. Because training does not occur in the claim, an improvement to training the machine learning model cannot be present. Examiner notes that Desjardins recited a particular method of training a machine learning model that resulted in an improvement to the machine learning model. However, assuming arguendo that the model was trained/being trained, there is nothing in the claim that improves the model by training the machine learning model. As noted in the updated 101 rejection above, all identified additional elements are recited at a high-level of generality (i.e., as a generic processor performing generic computer functions) such that it amounts to no more than mere instructions to apply the exceptions using a generic computer component. The claim is using a computer as a tool to perform the recited abstract idea and any improvement present is an improvement to the abstract idea of, to paraphrase, manage annotated medical data quality. Thus, a practical application is not present. (c) Even assuming arguendo (without conceding) that claim 1 is directed to an abstract idea under Step 2A, the claim recites "significant more" under Step 2B. As noted above, the claim recites improvements to technology, which supports a finding of "significantly more" under Step 2B. Furthermore, the claim does not attempt to preempt all ways of performing the alleged abstract idea, which further supports a finding of "significantly more" under Step 2B. A finding of "significantly more" under Step 2B is further supported based on the novelty and non-obviousness of the claim, as discussed below with respect to the prior art rejections. Therefore, claim 1 is patent-eligible under 35 U.S.C. § 101 for at least the foregoing reasons. (p. 14). Regarding (c), Examiner respectfully disagrees. MPEP 2106.04(I) states that "questions of preemption are inherent in and resolved by the two-part framework from Alice Corp. and Mayo (the Alice/Mayo test referred to by the Office as Steps 2A and 2B)." Thus, pre-emption concerns are fully addressed and made moot upon application of the two-part Alice Corp. subject matter eligibility analysis, as provided in the basis of rejection. Further, as noted in response to argument (b) above, the claims do not recite an improvement to technology and thus the claims do not provide an inventive concept (“significantly more”) under Step 2B. See updated 101 rejection above. (d) Independent claim 11 recites features similar, although not identical, to features recited in claim 1, and therefore is patent-eligible under 35 U.S.C. § 101 for at least similar reasons. Each of claims 2-8, 10, and 20-21 ultimately depends from claim 1, and therefore is patent- eligible under 35 U.S.C. § 101 for at least similar reasons to claim 1, and further in view of the additional features recited therein. (p. 14). Regarding (d), Examiner respectfully disagrees. Based on response to arguments above, claims 1 and 11 are unpatentable and therefore all claims depending therefrom are unpatentable according to the same rationale. See updated rejection above. Regarding rejections under 35 USC § 103 to Claims 1-8, 10-12, 14-18 and 20-22, Applicant’s arguments have been fully considered and are not all persuasive. The rejection has been updated in light of latest amendments. Applicant argues: (e) However, Abedin does not teach or suggest using annotations of a subset of medical data items randomly selected from a larger group of medical data items for verification of annotations of the larger group of medical data items. For example, claim 1 recites "randomly selecting, by the computing device, based on a sample size, and from the plurality of medical data items, a subset of the plurality of medical data items," "wherein the primary annotated data comprise annotations of the plurality of medical data items," "wherein the secondary annotated data are annotations of the subset of the plurality of medical data items," "comparing the primary annotated data with the secondary annotated data based on a verification condition using the computing device, wherein the verification condition includes a threshold related to an inter-observer agreement between different sets of annotated data for quality assurance of the primary annotated data," and "attaching verified status metadata to the primary annotated data when the comparison of the primary annotated data with the secondary annotated data satisfies the verification condition." Abedin does not teach or suggest at least these features. Therefore, claim 1 is distinguishable over the cited references. (p. 17). Regarding (e), Examiner respectfully disagrees. Examiner outlines each of the five specific limitations below argued by Applicant: (1) Claim 1 recites “ randomly selecting, by the computing device, based on a sample size, and from the plurality of medical data items, a subset of the plurality of medical data items; ”. Abedin discloses forming a collection of medical images is formed containing a subset ( selecting a subset of the plurality of medical data items ) of the plurality of medical images received ( based on a sample size, and from the plurality of medical data items ). However, Examiner agrees that Abedin fails to disclose “ randomly ” selecting a subset of data items. Therefore, the rejection has been withdrawn. However, upon further consideration, a new grounds of rejection necessitated by Applicant’s amendments is made in view of Ghosh et al. (US 6978458), as per the rejection above. (2) Claim 1 recites “ receiving, by the computing device, primary annotated data from the first client device over the network, wherein the primary annotated data comprise annotations of the plurality of medical data items generated at the first client device; ”. Abedin discloses providing the formed collections of medical images ( the plurality of medical data items ) to remote users and then receiving annotations ( receiving annotations of the plurality of medical data items generated at the first client device ) from each of the plurality of remote users ( primary annotated data generated at the first client device ). (3) Claim 1 recites ” receiving, by the computing device, secondary annotated data from the second client device over the network, wherein the secondary annotated data are annotations of the subset of the plurality of medical data items generated at the second client device; ”. Abedin discloses that the collection/the subset of the plurality of medical images are provided to the users and image annotations are then received ( receiving annotations of the subset of the plurality of medical data items generated at the second client device ) from each of the plurality of remote users ( secondary annotated data from the second client device ). (4) Claim 1 recites “ comparing the primary annotated data with the secondary annotated data based on a verification condition using the computing device… ”. Abedin discloses that the data model for a collection can handle multiple annotators across tasks, and multiple annotations for the same attribute by different annotators for cross-validation ( comparing the primary annotated data with the secondary annotated data based on a verification condition ). Claim 1 further recites “… wherein the verification condition includes a threshold related to an inter-observer agreement between different sets of annotated data for quality assurance of the primary annotated data ; ” and Examiner relies upon Diedrich to teach these limitations. Diedrich teaches inter-rater reliability which may include the ability to compare peer to peer quality for consistency to ensure that images are being interpreted at a higher quality ( for quality assurance of the primary annotated data ), a higher quality as ranked by the annotator ( an inter-observer agreement between different sets of annotated data ). The image quality threshold or the threshold ( the verification condition includes a threshold ) may be set based on set parameters, such as a percentage of the inter-rater reliability ( related to an inter-observer agreement between different sets of annotated data ). (5) Claim 1 recites “ attaching verified status metadata to the primary annotated data when the comparison of the primary annotated data with the secondary annotated data satisfies the verification condition; ”. Examiner notes that Abedin was not relied upon to disclose these limitations in the previous office action, but rather relied upon Diedrich and continues to rely upon Diedrich to teach these limitations in the updated rejection above. Diedrich teaches that inter-rater reliability may include the ability to compare peer to peer quality for consistency to ensure that images are being interpreted at a higher quality ( when the comparison of the primary annotated data with the secondary annotated data ). Further, Diedrich teaches that the quality measures may be published in multiple ways such as stored in a private tag in DICOM or published in the metadata ( attaching verified status metadata to the primary annotated data – para 57) because, e.g., higher market prices are determined for higher quality annotation data. If the annotation quality threshold has been met ( satisfies the verification condition ), then the annotation quality metrics are provided/stored to the marketplace ( attaching verified status ). Therefore, claim 1 and all claims depending therefrom remain rejected under 103. See updated rejection above. (f) Independent claim 11 recites features similar, although not identical, to features recited in claim 1, and therefore is distinguishable over the cited references for at least similar reasons. Each of claims 2-8, 10, and 20-21 ultimately depends from claim 1, and therefore is distinguishable over the cited references for at least similar reasons to claim 1, and further in view of the additional features recited therein. Each of claims 12, 14-18, and 22 ultimately depends from claim 11, and therefore is distinguishable over the cited references for at least similar reasons to claim 11, and further in view of the additional features recited therein. (p.17). Regarding (f), Examiner respectfully disagrees. Based on response to arguments above, claims 1 and 11 are unpatentable and therefore all claims depending therefrom are unpatentable according to the same rationale. See updated rejection above. Conclusion 07-40 AIA Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL . See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to KIMBERLY VANDER WOUDE whose telephone number is (703)756-4684. The examiner can normally be reached M-F 9 AM-5 PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PETER H CHOI can be reached at (469) 295-9171. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /K.E.V./Examiner, Art Unit 3681 /PETER H CHOI/Supervisory Patent Examiner, Art Unit 3681 Application/Control Number: 17/849,480 Page 2 Art Unit: 3681 Application/Control Number: 17/849,480 Page 3 Art Unit: 3681 Application/Control Number: 17/849,480 Page 5 Art Unit: 3681 Application/Control Number: 17/849,480 Page 6 Art Unit: 3681 Application/Control Number: 17/849,480 Page 7 Art Unit: 3681 Application/Control Number: 17/849,480 Page 8 Art Unit: 3681 Application/Control Number: 17/849,480 Page 9 Art Unit: 3681 Application/Control Number: 17/849,480 Page 10 Art Unit: 3681 Application/Control Number: 17/849,480 Page 11 Art Unit: 3681 Application/Control Number: 17/849,480 Page 12 Art Unit: 3681 Application/Control Number: 17/849,480 Page 13 Art Unit: 3681 Application/Control Number: 17/849,480 Page 14 Art Unit: 3681 Application/Control Number: 17/849,480 Page 15 Art Unit: 3681 Application/Control Number: 17/849,480 Page 16 Art Unit: 3681 Application/Control Number: 17/849,480 Page 17 Art Unit: 3681 Application/Control Number: 17/849,480 Page 18 Art Unit: 3681 Application/Control Number: 17/849,480 Page 19 Art Unit: 3681 Application/Control Number: 17/849,480 Page 20 Art Unit: 3681 Application/Control Number: 17/849,480 Page 21 Art Unit: 3681 Application/Control Number: 17/849,480 Page 22 Art Unit: 3681 Application/Control Number: 17/849,480 Page 23 Art Unit: 3681 Application/Control Number: 17/849,480 Page 24 Art Unit: 3681 Application/Control Number: 17/849,480 Page 25 Art Unit: 3681 Application/Control Number: 17/849,480 Page 26 Art Unit: 3681 Application/Control Number: 17/849,480 Page 27 Art Unit: 3681 Application/Control Number: 17/849,480 Page 28 Art Unit: 3681 Application/Control Number: 17/849,480 Page 29 Art Unit: 3681 Application/Control Number: 17/849,480 Page 30 Art Unit: 3681 Application/Control Number: 17/849,480 Page 31 Art Unit: 3681 Application/Control Number: 17/849,480 Page 32 Art Unit: 3681 Application/Control Number: 17/849,480 Page 33 Art Unit: 3681 Application/Control Number: 17/849,480 Page 34 Art Unit: 3681 Application/Control Number: 17/849,480 Page 35 Art Unit: 3681 Application/Control Number: 17/849,480 Page 36 Art Unit: 3681 Application/Control Number: 17/849,480 Page 37 Art Unit: 3681 Application/Control Number: 17/849,480 Page 38 Art Unit: 3681 Application/Control Number: 17/849,480 Page 39 Art Unit: 3681 Application/Control Number: 17/849,480 Page 40 Art Unit: 3681 Application/Control Number: 17/849,480 Page 41 Art Unit: 3681 Application/Control Number: 17/849,480 Page 42 Art Unit: 3681 Application/Control Number: 17/849,480 Page 43 Art Unit: 3681 Application/Control Number: 17/849,480 Page 44 Art Unit: 3681
Prosecution Insights