102Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This communication is a Non-Final Office Action in response to application number 19/102,209 received on 02/07/2025.
In accordance with Applicant’s filing, claims 1-21 are currently pending and have been examined.
Priority
Applicants claim for the benefit of a prior-filed application under 35 U.S.C. 119 and/or 35 U.S.C. 120 is acknowledged.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 02/07/2025, 05/06/2025, 06/24/2025, and 05/01/2026 have been considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-21 are rejected under 35 USC 101 because the claimed invention is directed to a judicial exception (i.e. abstract idea) without anything significantly more.
Step 1: The claimed invention is analyzed to determine if it falls outside one of the four statutory categories of invention. See MPEP 2106.03
Claims 1-18 are directed to a Method (i.e., Process), claim 20 is directed to a System (i.e., Machine), and claim 21 is directed to a non-transitory computer-readable medium (i.e., Manufacture). Therefore, claims 1-21 are directed to patent eligible categories of invention. Accordingly, the claims satisfy Step 1 of the eligibility inquiry.
Step 2A, Prong 1: In prong one of step 2A, the claim(s) is/are analyzed to evaluate whether they recite a judicial exception. See MPEP 2106.04
Independent claim 1 recites a method for gathering information about a process being performed by a user of a computing device. As drafted, the limitations recited by claim 1 fall under the “Mental Processes” abstract idea grouping by setting forth activities that could be performed mentally by a human (including an observation, evaluation, judgment, opinion, or with the help of pen and paper).
The abstract limitations of claim 1 include: “identifying a respective particular plurality of attributes to obtain multiple sets of attributes, the respective plurality of attributes including a first attribute, the identifying comprising: collecting: action information associated with zero, one or more actions performed by the user; and contextual information associated with one or more UI elements; and analyzing, the contextual information to identify the respective particular plurality of attributes, each of which corresponds to at least one respective UI element, the analyzing comprising: identifying, for each of the respective particular plurality of attributes, a respective attribute name, a respective attribute value, and/or a respective location in the particular UI screen, the identifying comprising: identifying a first value for the first attribute; and identifying, using the first value, a first name for the first attribute; and storing the multiple sets of attributes and information indicating names, values, and/or locations of attributes in the multiple sets of attributes.”. These limitations, as drafted, but for the recitation of additional elements in the claims, is a process that covers performance of the limitations in the mind. That is, nothing in the claim elements preclude the steps from practically being performed in the human mind.
Independent claims 20 and 21 recites a system and at least one non-transitory computer-readable medium for performing limitations that are substantially similar to those set forth in claim 1. Therefore, the same analysis applies to claims 20 and 21.
Dependent claims 9-11, and 18 further narrow the abstract idea and introduce further additional elements for consideration.
Dependent claims 2-8, 12-17 and 19 further narrow the abstract idea and do not introduce any additional elements for consideration. In other words, each of the limitations/elements recited in respective dependent claims is/are further part of the abstract ideas as identified by the Examiner for each respective dependent claim (i.e., they are part of the abstract idea recited in each respective claim).
Step 2A, Prong 2: An evaluation is made whether a claim recites any additional element, or combination of additional elements, that integrate the judicial exception into a practical application of the exception. See MPEP 2106.04(d).
Regarding the computing additional elements, namely a computing device having computer software programs and separate monitoring software installed, at least one non-transitory computer-readable medium, analyzing, using at least one processor, collecting with the monitoring software executing on the computing device, for each particular UI screen of at least some of the UI screens in the sequence of UI screens, one or more actions performed by the user via the particular UI screen, contextual information associated with one or more UI elements visible in the particular UI screen, and each of which corresponds to at least one respective UI element visible in the particular UI screen from independent claims 1/20/21, these additional elements have been evaluated but fail to integrate the abstract idea into a practical application because they amount to using generic computing elements or instructions (software) to perform the abstract idea, similar to adding the words “apply it” (or equivalent), which merely serves to link the use of the judicial exception to a particular technological environment (generic computing environment). See MPEP 2106.05(f) and 2106.05(h).
Therefore, the additional elements of the independent claims, when considered both individually and in combination, are not sufficient to prove integration into a practical application.
Regarding the additional elements wherein the computer software programs comprise an Internet browser, and using a document object model (DOM) representation of a webpage displayed via the Internet browser from claim 9, using network application programming interface (API) requests sent and/or received by the Internet browser from claim 10, and a desktop application from claim 11, these additional elements have been evaluated but fail to integrate the abstract idea into a practical application because they amount to using generic computing elements or instructions (software) to perform the abstract idea, similar to adding the words “apply it” (or equivalent), which merely serves to link the use of the judicial exception to a particular technological environment (generic computing environment). See MPEP 2106.05(f) and 2106.05(h).
Dependent claims 2-8, 12-17 and 19 recite the same abstract ideas (“mental processes”) as the independent claims along with further steps/details falling under the scope of the abstract idea itself, along with the same or substantially same additional elements addressed.
Accordingly, because the Step 2A Prong One and Prong Two analysis resulted in the conclusion that the claims are directed to an abstract idea, additional analysis under Step 2B of the eligibility inquiry must be conducted in order to determine whether any claim element or combination of elements amount to significantly more than the judicial exception.
Step 2B: The claims are analyzed to determine whether any additional element, or combination of additional elements, is/are sufficient to ensure that the claims amount to significantly more than the judicial exception. This analysis is also termed a search for "inventive concept." See MPEP 2106.05.
Regarding the computing additional elements, namely a computing device having computer software programs and separate monitoring software installed, at least one non-transitory computer-readable medium, analyzing, using at least one processor, collecting with the monitoring software executing on the computing device, for each particular UI screen of at least some of the UI screens in the sequence of UI screens, one or more actions performed by the user via the particular UI screen, contextual information associated with one or more UI elements visible in the particular UI screen, and each of which corresponds to at least one respective UI element visible in the particular UI screen from independent claims 1/20/21, these additional elements have been evaluated but fail to add significantly more to the claims because they amount to using generic computing elements or instructions (software) to perform the abstract idea, similar to adding the words “apply it” (or equivalent), which merely serves to link the use of the judicial exception to a particular technological environment (generic computing environment). See MPEP 2106.05(f) and 2106.05(h). Therefore, the computing additional elements merely describe generic computing elements or computer-executable instructions (software) merely serve to tie the abstract idea to a particular operating environment, which does not add significantly more to the abstract idea. See, e.g., Alice Corp., 134 S. Ct. 2347, 110 USPQ2d 1976; Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015).
Therefore, the additional elements of the independent claims, when considered both individually and in combination, are not sufficient to add significantly more to the claims.
With respect to the additional elements wherein the computer software programs comprise an Internet browser, and using a document object model (DOM) representation of a webpage displayed via the Internet browser from claim 9, using network application programming interface (API) requests sent and/or received by the Internet browser from claim 10, and a desktop application from claim 11, these additional elements have been evaluated but fail to add significantly more to the claims because they amount to using generic computing elements or instructions (software) to perform the abstract idea, similar to adding the words “apply it” (or equivalent), which merely serves to link the use of the judicial exception to a particular technological environment (generic computing environment). See MPEP 2106.05(f) and 2106.05(h). Therefore, the computing additional elements merely describe generic computing elements or computer-executable instructions (software) merely serve to tie the abstract idea to a particular operating environment, which does not add significantly more to the abstract idea. See, e.g., Alice Corp., 134 S. Ct. 2347, 110 USPQ2d 1976; Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015).
Therefore, the additional elements of the independent claims, when considered both individually and in combination, are not sufficient to add significantly more to the claims.
Dependent claims 2-8, 12-17 and 19 recite the same abstract ideas (“mental processes”) as the independent claims along with further steps/details falling under the scope of the abstract idea itself, along with the same or substantially same additional elements addressed above under Step 2A Prong Two and Step 2B, which is incorporated herein.
The ordered combination of elements in the dependent claims (including the limitations inherited from the parent claim(s)) add nothing that is not already present as when the elements are taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide generic computer implementation. Accordingly, the subject matter encompassed by the dependent claims fails to amount to a practical application or significantly more than the abstract idea itself.
Accordingly, claims 1-21 are rejected under 35 USC 101.
Claim Rejections - 35 USC § 103
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7, 8, 12, 17, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Dicker et al. (US 8965998 B1, hereinafter “Dicker”), in view of da Veiga et al. (US 20160027216 A1, hereinafter “da Veiga”).
Regarding claims 1/20/21: Dicker teaches:
a method for gathering information about a process being performed by a user of a computing device ([Column 7, Lines 53-54] a method 500 for determining component scores, which are used in selecting subsets);
a system for gathering information about a process being performed by a user of a computing device ([Column 7, Lines 44-45] a system 400 for selecting subsets of components and serving web pages);
the analyzing comprising: identifying, for each of the respective particular plurality of attributes, a respective attribute name, a respective attribute value, and/or a respective location in the particular UI screen, ([Column 10, Lines 4-12] As discussed above with respect to step 506, for each component in the set, user activity related to exposures of each component is monitored. As will be understood by one skilled in the art, different kinds and types of user activity can be measured depending upon the application. For an information-related web site, traversals of hypertext links may be the only activity of interest. For an on-line merchant, however, addition of products to a shopping cart, and/or the purchase of products may be actions of interest.);
the identifying comprising: identifying a first value for the first attribute; ([column 6, Lines 31-36] the web page 200 and the set of possible components are associated with a single context. The context can include a single attribute, such as, for example, the value of the path of this web page in the web site hosting the page: web page identifier: /homepage.html);
and identifying, using the first value, a first name for the first attribute; (([column 6, Lines 31-36] the web page 200 and the set of possible components are associated with a single context. The context can include a single attribute, such as, for example, the value of the path of this web page in the web site hosting the page: web page identifier: /homepage.html; [Column 6, Lines 37-51] two or more contexts can be used for selecting components for the web page 200 by using attributes that may depend upon the identity of the user. For example, a first context can include the following attributes: web page identifier: /homepage.html last purchase >1 year ago: Y A second context can include the following attributes: web page identifier: /homepage.html last purchase >1 year ago: N Any number of contexts can be configured to be used in conjunction with a web page. Different contexts can be associated with different user characteristics (e.g., new user, frequent purchaser, electronics buff) or other variables, such as, for example, time of day (e.g., morning, afternoon, evening), or time of year (e.g., summer, winter, Christmas).);
and storing the multiple sets of attributes and information indicating names, values, and/or locations of attributes in the multiple sets of attributes. ([Column 8, Lines 26-46] The database 408 preferably includes entries for one or more contexts 110. For each context 110, data is preferably maintained for each of several components 410, all of which make up a set or list. For each component 410, exposure data 412, activity data 105 and a component score 416 are preferably maintained.; The database 408 preferably also maintains component value data 418 for each component. The value data can represent a value or benefit to an entity operating a web site that results from user activity related to a component. For example, for an on-line merchant, a value for a component can be representative of a margin, profit, or contribution associated with a user's purchase of a product identified by the component. In the illustrated system 400 the value data 418 for each component is not necessarily associated with the context or contexts that include the component. In an alternative embodiment, the component value data is maintained for each component in association with a corresponding context. This value data 418, which will be discussed in greater detail below, can be used in determining a component's activity data 105 or score 416.).
Dicker doesn’t teach:
at least one non-transitory computer-readable medium for gathering information about a process being performed by a user of a computing device
for each particular UI screen of at least some of the UI screens in the sequence of UI screens, identifying a respective particular plurality of attributes to obtain multiple sets of attributes, the respective plurality of attributes including a first attribute,
the identifying comprising: collecting with the monitoring software executing on the computing device: action information associated with zero, one or more actions performed by the user via the particular UI screen;
and contextual information associated with one or more UI elements visible in the particular UI screen;
and analyzing, using at least one processor, the contextual information to identify the respective particular plurality of attributes, each of which corresponds to at least one respective UI element visible in the particular UI screen,
da Veiga teaches:
at least one non-transitory computer-readable medium for gathering information about a process being performed by a user of a computing device ([0005] an article of manufacture such as one or more computer-readable storage media.);
for each particular UI screen of at least some of the UI screens in the sequence of UI screens, identifying a respective particular plurality of attributes to obtain multiple sets of attributes, the respective plurality of attributes including a first attribute, ([Fig. 6] Step 630: UI client calculates initial cursor position based on exit point on PC monitor screen.);
the identifying comprising: collecting with the monitoring software executing on the computing device: action information associated with zero, one or more actions performed by the user via the particular UI screen; ([0037] In step 625, the UI server informs the UI client that the mouse is operating in the virtual world and it passes mouse messages such as mouse movements and user inputs (e.g., button clicks, scroll wheel actions, etc.) to the UI client.);
and contextual information associated with one or more UI elements visible in the particular UI screen; ([0037] The cursor may be dynamically rendered in 3D using a size that is proportional to the cursor's distance from the user in the viewport. That is, it is typically rendered to be bigger when it is closer to the viewer and smaller when it is farther away.);
and analyzing, using at least one processor, the contextual information to identify the respective particular plurality of attributes, each of which corresponds to at least one respective UI element visible in the particular UI screen, ([0046] In step 1215, when the mouse movement indicates that the cursor is moving off the edge of the monitor, then the UI server takes control of the mouse messages and prevents them from propagating to other systems that are running on the device.);
It would’ve been obvious to one of ordinary skill in the art at the time of Applicant’s invention, to combine Dicker with da Veiga’s features listed above. One would’ve been motivated to do so in order to determine where the user is looking so that user inputs are appropriately directed to the monitor or viewport (da Veiga; [Abstract]). By combining the teachings of da Veiga, one would’ve been able to successfully monitor user interactions with UI elements.
Regarding Claim 2: Dicker teaches:
collecting contextual information associated with at least one UI element that the user does not interact with when performing actions via the particular UI screen. ([Column 9, Lines 8-13] At a step 608, in one embodiment, the component selection module 422 optionally swaps into the subset 106 one or more components that have not been selected. The swapping of unselected components into the subset 106 provides an opportunity for activity to be measured for components with little or no prior exposure.).
Regarding Claim 3: Dicker teaches:
collecting contextual information associated with at least one non-interactive UI element visible in the particular UI screen. ([Column 9, Lines 8-13] At a step 608, in one embodiment, the component selection module 422 optionally swaps into the subset 106 one or more components that have not been selected. The swapping of unselected components into the subset 106 provides an opportunity for activity to be measured for components with little or no prior exposure.).
Regarding Claim 4: Dicker teaches:
wherein collecting the action information comprises collecting action information associated with one or more actions performed via the particular UI screen. ([Column 8, Lines 55-63] FIG. 6 illustrates a method 600 for responding to a request for a web page by selecting components from a set in accordance with one embodiment. At a step 602 the web server 402 receives a web page request from a user 404. At a step 604, the web server 402 identifies an associated context, which may be based upon the page request, the user's web browsing session and/or data related to the user. The method 600 is preferably invoked in response to certain types of pages that are configured to utilize subsets of components.).
Regarding Claim 5: Dicker teaches:
identifying, using the first value and an object hierarchy including objects corresponding to UI elements of the particular UI screen, the first name for the first attribute. ([Column 3, Lines 23-24] a web page identifier that identifies the web page or type of web page the user is browsing.).
Regarding Claim 7: Dicker teaches:
when the first attribute is determined to be an attribute that has not been previously stored: storing the first value for the first attribute in a first data structure, ([Column 12, Lines 22-39] values of state variables are coded into bit fields in a binary number and the resulting number defines and identifies the context. For example, for an on-line merchant, attributes can be encoded as bit field elements as follows: 1=recognized customer 2=last purchase more than 1 year ago 4=purchased more than 10 components 8=shopping cart contains more than $100 16=shopping cart contains between $50 and $100 32=order contains gift components A customer that has only been a customer for three months, has purchased twelve components in that time, and has a $60 book selected as a gift in his shopping cart will be in the context represented by the number 53. By encoding attributes as bit fields, the context can be easily be determined on the fly by testing each of the aforementioned conditions and adding together the resulting values.; [Column 2, Lines 25-36] The context associated with a particular dynamically-generated web page may optionally reflect the browsing and/or purchase histories of users of a web site, such that components presented on that web page over the same time period vary from user to user. For example, if the current visitor to the web page is a frequent customer of the web site, a component may automatically be selected that has frequently produced a desirable result (e.g., an item purchase) when presented to other frequent customers on that web page. On the other hand, new customers who access the same web page may be presented with a different component--one that has been particularly effective when presented to new customers.);
and storing the first name for the first attribute in a second data structure different from the first data structure. ([Column 11, Lines 58-61] The state variables 710, which can be used to store data about the user and/or the user's browsing session are preferably maintained and updated by the web server system 400 for each user.).
Regarding Claim 8: Dicker teaches:
when the first attribute is determined to be an attribute that has been previously stored, storing the first value for the first attribute in the first data structure. ([Fig. 7] 720A: Context A Attributes, 720B: Context B Attributes; [Column 12, Lines 22-39] values of state variables are coded into bit fields in a binary number and the resulting number defines and identifies the context. For example, for an on-line merchant, attributes can be encoded as bit field elements as follows: 1=recognized customer 2=last purchase more than 1 year ago 4=purchased more than 10 components 8=shopping cart contains more than $100 16=shopping cart contains between $50 and $100 32=order contains gift components A customer that has only been a customer for three months, has purchased twelve components in that time, and has a $60 book selected as a gift in his shopping cart will be in the context represented by the number 53. By encoding attributes as bit fields, the context can be easily be determined on the fly by testing each of the aforementioned conditions and adding together the resulting values.).
Regarding Claim 12: Dicker teaches:
collecting the action information and the contextual information by accessing a memory space of a computer software program, of the computer software programs, that generated the particular UI screen. ([Column 4, Lines 14-30] a context defines an environment in association with which data is collected and in association with which components are selected for inclusion on web pages based upon the collected data. A context can be identified or defined through one or more attributes or values descriptive of, related to, and/or identifying, for example, (a) a user, and/or (b) a state of a user's browsing session. The attributes can be web browsing session specific values, such as the location of the current web page the user is requesting or the locations of one or more previous web pages requested by the user. The attributes can also be non-session specific values such as the identity of a user, what the user has in an electronic shopping cart, the past purchase history of the user, or how long the user has been a customer of an on-line merchant.).
Regarding Claim 17: Dicker teaches:
identifying a first value for the first attribute when performing a first sequence of actions via a first UI screen at the computing device; ([Column 10, Lines 13-17] multiple types of actions are monitored and weighted in determining a measure of activity. For an on-line merchant, for example, the types of actions tracked might include (a) traversals of component-related links);
identifying a second value for the first attribute when performing a second sequence of actions via a second UI screen at the computing device; ([Column 10, Lines 13-17] multiple types of actions are monitored and weighted in determining a measure of activity. For an on-line merchant, for example, the types of actions tracked might include (a) traversals of component-related links, multiple types of actions are monitored and weighted in determining a measure of activity. For an on-line merchant, for example, the types of actions tracked might include (a) traversals of component-related links, (b) addition of component-related product(s) to a wish list);
and determining, when the first and second values of the first attribute are the same, that the first sequence of actions and the second sequence of actions belong to the same process. ([Column 2, Lines 34-36] new customers who access the same web page may be presented with a different component--one that has been particularly effective when presented to new customers. Examiner notes that one of ordinary skill in the art would reasonably interpret the action of displaying a component to a new customer who accesses a web page based on the success of a similar component displayed to a previous new customer who accessed the same web page, as equivalent to determining that the actions belong to the same process (i.e., new customer accessing the web page.).).
Regarding Claim 18: Dicker teaches:
displaying, via a graphical user interface, the multiple sets of attributes and information indicating at least the names and values of the attributes in the multiple sets of attributes. ([Column 6, Lines 53-67] FIG. 3 illustrates another example web page 300 from a merchant web site. On the left, the web page 300 shows an electronic shopping cart 302 to which one selection 304 has been added. To the right of the shopping cart 302, the web page shows several components. The components of interest in this case are the three components of the web page 311-313 titled: "Customers who bought [the product added to the shopping cart] also bought:" "Quick Picks:" "Top Sellers in [the category of the product added to the shopping cart]:" In this case, each of the components 311-313 identifies different types of products that the user might be interested in purchasing.).
Claims 6, 9-11, and 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Dicker et al. (US 8965998 B1, hereinafter “Dicker”), in view of da Veiga et al. (US 20160027216 A1, hereinafter “da Veiga”) as applied to claims 1/5/20/21, in further view of Arthursson (US 20090172101 A1, hereinafter “Arthursson”).
Regarding Claim 6: Dicker doesn’t teach:
identifying, in the object hierarchy, a location of a first object corresponding to a first UI element representing the first value;
identifying, in the object hierarchy, a location of a second object corresponding to a second UI element representing the first name;
and determining that the first name is associated with the first value when the first object and the second object are located within a threshold distance of each other in the object hierarchy.
Arthursson teaches:
identifying, in the object hierarchy, a location of a first object corresponding to a first UI element representing the first value; ([0363] FIG. 43 illustrates an example collaboration application easily created using embodiments of the system. The Class of 1992 Reunion application is one example of functionality that could be presented to members of a group. As described above with respect to FIG. 42, FIG. 43 illustrates a collection of components that refer to multiple data sources. What is illustrated by the Class of 1992 Reunion application is a collection of components that could be launched as part of an autostart.xml file loaded when binding a group folder. As shown above in FIG. 32B, the Class of 1992 Reunion group folder may be bound by a client as a data source, in which case it would be displayed as a drive within the user interface.; [0372] the state information serialized at block 4510 may include variables that describe the condition of an application's user interface components (e.g., minimized, expanded, selected, etc.), dynamically loaded resources, coordinate positions occupied by an application on a visual display, etc.);
identifying, in the object hierarchy, a location of a second object corresponding to a second UI element representing the first name; ([0363] FIG. 43 illustrates an example collaboration application easily created using embodiments of the system. The Class of 1992 Reunion application is one example of functionality that could be presented to members of a group. As described above with respect to FIG. 42, FIG. 43 illustrates a collection of components that refer to multiple data sources. What is illustrated by the Class of 1992 Reunion application is a collection of components that could be launched as part of an autostart.xml file loaded when binding a group folder. As shown above in FIG. 32B, the Class of 1992 Reunion group folder may be bound by a client as a data source, in which case it would be displayed as a drive within the user interface.; [0372] the state information serialized at block 4510 may include variables that describe the condition of an application's user interface components (e.g., minimized, expanded, selected, etc.), dynamically loaded resources, coordinate positions occupied by an application on a visual display, etc.);
and determining that the first name is associated with the first value when the first object and the second object are located within a threshold distance of each other in the object hierarchy. ([0279] FIG. 26 illustrates one embodiment of some of the details of the content of the message server 2512. The message server 2512 contains a subscription list for each client using the XML file system. For example, FIG. 26 illustrates two clients: client one 2606 and client two 2608. Client one 2606 is associated with a client one subscription list 2602, and client two 2608 is associated with a client two subscription list 2604.).
It would’ve been obvious to one of ordinary skill in the art at the time of Applicant’s invention, to combine modified Dicker with Arthursson’s features listed above. One would’ve been motivated to do so in order to include variables that describe the condition of an application's user interface components (Arthursson; [0372]). By combining the teachings of Arthursson, one would’ve been able to successfully identify the positions of objects in a user interface.
Regarding Claim 9: Dicker teaches:
wherein the computer software programs comprise an Internet browser, ([Column 2, Lines 9-13] The set of components, the details of the selection process, the activity data and the web page are preferably associated with a context representing a state of a user and/or a user browsing session.; [Column 14, Lines 49-50] web pages are larger in size than can be displayed in a web browser window without scrolling);
Dicker doesn’t teach:
and collecting the action information and the contextual information further comprises: collecting the action information and the contextual information using a document object model (DOM) representation of a webpage displayed via the Internet browser.
Arthursson teaches:
and collecting the action information and the contextual information further comprises: collecting the action information and the contextual information using a document object model (DOM) representation of a webpage displayed via the Internet browser. ([0189] document objects serve as a wrapper to DOM ("Document Object Model") objects utilized by a Web browser and an XML parser. In this regard, enhanced features are encoded within the document object provided by the invention that includes the ability to add any objects that exist within the network operating system environment as listeners for updates made to the data model.).
It would’ve been obvious to one of ordinary skill in the art at the time of Applicant’s invention, to combine modified Dicker with Arthursson’s features listed above. One would’ve been motivated to do so in order to maintain a list of listeners that will be notified in response to a data update (Arthursson; [0189]). By combining the teachings of Arthursson, one would’ve been able to successfully collect data using a DOM of a webpage.
Regarding Claim 10: Dicker doesn’t teach:
collecting the action information and the contextual information using network application programming interface (API) requests sent and/or received by the Internet browser.
Arthursson teaches:
collecting the action information and the contextual information using network application programming interface (API) requests sent and/or received by the Internet browser. ([0069] Using an Application Programming Interface (API), a communicator may be created that abstracts the data handling functions for interacting with any (internal or external) network service. By way of example, developers can create communicators that access network servers hosting XML Web services, REST services, XML resources, RSS or Atom feeds, text, csv text, HTML (Hypertext Markup Language) based Web sites, among others. Referring to FIG. 2, an instance of a communicator or "channel" may be instantiated by the client 212 in order to interact with the Web service 218. In this example, network operating system services are accessible on a public network (i.e., the Internet 214) to the client 212 using the server-side data center 216 as a proxy. Moreover, the Web service 218 is accessible to the clients 206-210 using a communicator even though network operating services are being provided from a private network (e.g., the enterprise network 204). In this instance, the server-side data center 216 serves as the proxy that manages communications between the clients 206-210 and the Web service 218. Accordingly, clients may use communicators to abstract data handling functions when accessing network services.).
It would’ve been obvious to one of ordinary skill in the art at the time of Applicant’s invention, to combine modified Dicker with Arthursson’s features listed above. One would’ve been motivated to do so in order to simplify application development since developers are not required to repetitively write code for managing communications between a client and network service (Arthursson; [0069]). By combining the teachings of Arthursson, one would’ve been able to successfully an API for communication.
Regarding Claim 11: Dicker teaches:
and the contextual information further comprises: collecting the action information and the contextual information by tracking events indicating changes to a structure of the particular UI screen. ([Column 1, Lines 40-55] in order to determine the effectiveness of presenting a component to a user, web site operators manually set up tests in which components are presented to users and activity resulting from presenting the components is tracked. The tracked activity can include any user activity of interest resulting from presenting the component to a user, such as a selection of a hypertext link included in the component, or an addition of a product displayed or represented by the component to a shopping cart or a wish list. The tests are typically conducted in such a way that users are not aware that they are the subject of a test of the effectiveness of a component. Based upon analysis of the resulting activity, the effectiveness of components can be determined. Determinations as to which components to present to users can then be based upon the determined effectiveness of the tested components.).
Dicker doesn’t teach:
wherein the computer software programs comprise a desktop application and collecting the action information
Arthursson teaches:
wherein the computer software programs comprise a desktop application and collecting the action information ([0326] the startup application is executed, and then executes a second application, such as a desktop application; [0328] A group, upon creation, stores a collection of group information).
It would’ve been obvious to one of ordinary skill in the art at the time of Applicant’s invention, to combine modified Dicker with Arthursson’s features listed above. One would’ve been motivated to do so in order to display a desktop to the user (Arthursson; [0326]). By combining the teachings of Arthursson, one would’ve been able to successfully use a desktop application.
Regarding Claim 13: Dicker doesn’t explicitly teach:
configuring, in an object hierarchy including objects corresponding to UI elements of the particular UI screen, one or more paths for obtaining the action information and the contextual information;
and collecting the action information and the contextual information by using the configured paths.
Arthursson teaches:
configuring, in an object hierarchy including objects corresponding to UI elements of the particular UI screen, one or more paths for obtaining the action information and the contextual information; ([0138] Through the controlled instantiation and referencing of objects, a localized relationship hierarchy may be established that delimits the boundary of an application instance. As described in further detail below, this framework associates process and view objects with a corresponding instance and delimits access to these objects from outside the localized relationship hierarchy.; [0162] the expression selected for evaluation at block 1203 is evaluated into an XBind at block 1204. As used herein, an XBind is a data type that comprises a URL, base path (e.g., an XPath expression that references an XML fragment within the document identified by the specified URL), and a selection (e.g., a plurality of XPath expressions).);
and collecting the action information and the contextual information by using the configured paths. ([0117] a binding provides an automated communication path between a user interface component and the underlying data model.).
It would’ve been obvious to one of ordinary skill in the art at the time of Applicant’s invention, to combine modified Dicker with Arthursson’s features listed above. One would’ve been motivated to do so in order to allow binding to be shared and/or transferred between user interface components (Arthursson; [0117]). By combining the teachings of Arthursson, one would’ve been able to successfully collect interaction data through configured paths.
Regarding Claim 14: Dicker doesn’t teach:
collecting first contextual information associated with UI elements visible in the particular UI screen and second contextual information associated with UI elements not visible in the particular UI screen;
and filtering, from the collected first and second contextual information, the second contextual information associated with the UI elements not visible in the particular UI screen.
Arthursson teaches:
collecting first contextual information associated with UI elements visible in the particular UI screen and second contextual information associated with UI elements not visible in the particular UI screen; ([0367] As mentioned previously, the state of every component in the system may constantly be propagated to a system-provided state XML document utilizing the functionality encapsulated in a state manager 4412.; [0377] the state manager 4412 provides functionality to monitor the state of each user interface component in the system; [0105] Accordingly, the Button2 component 704 is depicted in FIG. 7A with a dashed line to indicate that the Button2 component 704 is not initially visible to the user subsequent to execution of the Action operation 614.; [0384] In another embodiment, an application involved in a collaboration session is "cloned" on each client. When this type of collaboration session is initiated, each client is provided with a distinct and separate application view. Regardless of whether the application view is shared or cloned, data updates and state synchronization are constantly propagated between clients involved in the collaboration session.);
and filtering, from the collected first and second contextual information, the second contextual information associated with the UI elements not visible in the particular UI screen. ([0142] The relationship between objects depicted in FIG. 10B illustrate how the present invention is able to ensure intra-application security. Access to application objects is controlled by the application manager 904 which exposes methods for creating, opening, and terminating applications. When an application object is instantiated, the application manager 904 isolates the application object into a separate memory space. By preventing application objects from sharing a memory space, code from one application may not be injected into or otherwise affect the memory space allocated to a different application. Moreover, an application object provides an encapsulated representation of an application in which internal data associated with the application may be hidden. All of the functionality of the application and access to internal data is controlled through the creation of exposed methods. By isolating and encapsulating applications in this way, the internal objects (e.g., view objects, instance object, process object, etc.) associated with one application are rendered inaccessible to a different application.).
It would’ve been obvious to one of ordinary skill in the art at the time of Applicant’s invention, to combine modified Dicker with Arthursson’s features listed above. One would’ve been motivated to do so, so that internal data of an application may not be accessed (Arthursson; [0142]). By combining the teachings of Arthursson, one would’ve been able to successfully collect data associated with elements not visible in the UI screen.
Regarding Claim 15: Dicker doesn’t teach:
generating a fingerprint of the particular UI screen,wherein generating the fingerprint comprises concatenating or hashing the attribute names associated with the respective plurality of attributes;
and using the fingerprint of the particular UI screen to identify a type of the process being performed by the user of the computer device.
Arthursson teaches:
generating a fingerprint of the particular UI screen, ([0171] the view manager 908 is called to create a view object that will be used to render a new user interface or application view.);
wherein generating the fingerprint comprises concatenating or hashing the attribute names associated with the respective plurality of attributes; ([0156] the event manager 914 maintains an internal hash data structure that associates a set of trigger data with listening notifier objects);
and using the fingerprint of the particular UI screen to identify a type of the process being performed by the user of the computer device. ([0151] processing is performed by the data type recognizer to identify the file-type of the document that will be opened. In this example, the analysis performed by the data type recognizer will determine that the document associated with the received call is a process XML document. As mentioned previously, the data type recognizer may associate actions with a particular file type).
It would’ve been obvious to one of ordinary skill in the art at the time of Applicant’s invention, to combine modified Dicker with Arthursson’s features listed above. One would’ve been motivated to do so in order to allow binding to be shared and/or transferred between user interface components (Arthursson; [0117]). By combining the teachings of Arthursson, one would’ve been able to successfully collect interaction data through configured paths.
Claims 16 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Dicker et al. (US 8965998 B1, hereinafter “Dicker”), in view of da Veiga et al. (US 20160027216 A1, hereinafter “da Veiga”) as applied to claims 1/5/20/21, in further view of Zohar et al. (US 10713068 B1, hereinafter “Zohar”).
Regarding Claim 16: Dicker doesn’t teach:
generating training data to be used for identification of one or more instances of the process,
the training data including the multiple sets of attributes and the information indicating the names, values, and/or locations of the attributes in the multiple sets of attributes as part of training data.
Zohar teaches:
generating training data to be used for identification of one or more instances of the process, ([Column 24, Lines 15-21] On Step 510, a training dataset may be compiled based on the labeled resolution. Each labeled resolution may be represented by a valuation of a feature vector and a corresponding label. The labeled dataset may be used to train a machine learning model to provide predictions and labeling for other situations.);
the training data including the multiple sets of attributes and the information indicating the names, values, and/or locations of the attributes in the multiple sets of attributes as part of training data. ([Column 8, Lines 17-28] the machine learning model may be configured to receive a feature vector and provide a label. In some exemplary embodiments, the feature vector may comprise a first sub-vector representing a first GUI, a second sub-vector representing a second GUI, and a third sub-vector representing the element of interest in the first GUI. Additionally, or alternatively, the third sub-vector may provide the properties used to represent the element of interest (e.g., the representation, a representation set, or the like). In some exemplary embodiments, the label may be a label indicating the properties or identity of the element of interest in the second GUI. Examiner notes that one of ordinary skill in the art would reasonably consider identity of the element as equivalent to a name.).
It would’ve been obvious to one of ordinary skill in the art at the time of Applicant’s invention, to combine modified Dicker with Zohar’s features listed above. One would’ve been motivated to do so in order to utilize a machine learning model to learn, based on a training dataset, how to acquire an element in a new GUI based on an existing GUI and an identification of the element therein (Zohar; [Column 8, Lines 13-16]). By combining the teachings of Zohar, one would’ve been able to successfully generate training data that includes the name of the attributes.
Regarding Claim 19: Dicker doesn’t teach:
determining, for each attribute in the multiple sets of attributes, a quality metric indicating a quality of the identification of the attribute;
and wherein the displaying comprises displaying, for at least some of the attributes in the multiple sets of attributes, the respective attribute names, the respective attribute values, and the respective quality metrics.
Zohar teaches:
determining, for each attribute in the multiple sets of attributes, a quality metric indicating a quality of the identification of the attribute; ([Column 8, Lines 33-35] The label may provide a confidence measurement whether the candidate element corresponds to the element of interest);
and wherein the displaying comprises displaying, for at least some of the attributes in the multiple sets of attributes, the respective attribute names, the respective attribute values, and the respective quality metrics. ([Column 6, Lines 44-60] Another technical problem dealt with by the disclosed subject matter is to provide a robust acquisition process of a GUI element in a program that changes over time. A program may change over time as a result of adding new features, removing features, design changes, bug fixes, or the like. It may be desired to acquire elements robustly, so as to find the same element over time. As an example, an application may have a “Pay Now” button. The application may change over time. For example, the “Pay Now” button may change place within the application, the size of the button may change, the text may change, or the like. It may be desired to collect usage information automatically regarding how many users have pressed the pay button, how many users have completed the payment, how long did it take for a user to complete the purchase, or the like. Such collected data may be used, for example, in providing useful funnel analysis of the program. Having a representation of the payment button may allow to acquire the payment button in the GUI, and identify interactions therewith.).
It would’ve been obvious to one of ordinary skill in the art at the time of Applicant’s invention, to combine modified Dicker with Zohar’s features listed above. One would’ve been motivated to do so in order to allow to acquire the payment button in the GUI, and identify interactions therewith (Zohar; [Column 6, Lines 60-62]). By combining the teachings of Zohar, one would’ve been able to successfully present attribute names, values and the respective quality metric.
Accordingly, claims 1-21 are rejected under 35 USC 103.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Marggraff et al. (WO 2022093839 A1), which discloses systems and methods are provided herein for utilizing one or more human interaction entities (HIEs) to monitor and modulate an individual’s emotional state, as well as to encourage and/or enhance both structured (e.g., formal, institutional, certification-based) and non-structured learning.
N. Kussul and S. Skakun, "Neural network approach for user activity monitoring in computer networks," 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541), Budapest, Hungary, 2004, pp. 1557-1561 vol.2, which discloses a system for user activity monitoring in computer networks.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GABRIEL J TORRES CHANZA whose telephone number is (571)272-3701. The examiner can normally be reached Monday thru Friday 8am - 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Epstein can be reached on (571)270-5389. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/G.J.T./Examiner, Art Unit 3625
/BRIAN M EPSTEIN/Supervisory Patent Examiner, Art Unit 3625
Prosecution Insights