DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 12-17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 12 recites, “display the dataset through the surface display and the augmented reality (AR) wearbale display”. The phrase “the dataset” has a lack of antecedent basis and as a result the limitation’s scope is not clear.
Dependent claims 13-17 are also rejected based on dependency.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-18 are rejected under 35 U.S.C. 103 as being unpatentable over Mueller et al. ( WO 2020010350A, “Mueller”) in view of Hubenschmid et al. ("STREAM: Exploring the Combination of Spatially-Aware Tablets with Augmented Reality Head-Mounted Displays for Immersive Analytics "CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (May 2021)
Regarding claim 12, Mueller teaches, A system (Fig. 20), comprising a surface display ( element 112), the surface display comprising processing circuits and a memory, the memory containing instructions executable by the processing circuits whereby the system is operative to (“(figures 1, 1B-1F, 7-8C, 12, 17, 18; paragraphs [00025], [00079]-[00087], [000102], [000111], [000129]):
display the dataset through the surface display, the dataset comprising a plurality of variables; (figures 1, 1C, 6A, 7, 7E, 8B, 12, 17, 18; paragraphs [00040], [000100], [000120], [000160], [000165]) (paragraphs [000309], [000310], [000331])
the dataset comprising a plurality of variables (paragraphs [00036], [00091], [000147], [000148], [000156], [000286]) [00036] FIGS. 5 A, 5E, and 51 are the SHDs of rebuilt causal networks by binning numeric variables with different levels. FIGS. 5B, 5F and 5J are the Structure Hamming Distances
(SHDs) from GM and GM+UB with different numbers of categorical variables included in the dataset. FIGS. 5C, 5G and 5K show the average TPR. FIGS. 5D, 5H and 5L show the average TDR of the reconstructed networks with the three strategies under different numbers of categorical variables. Each of FIGS. 5A-5L provide respective illustrations of experimental results, each in accordance with an embodiment of the disclosed system and method.
receive a selection of variables, from the plurality of variables; ((Figure 1E; paragraphs [00079], [00080], [00086], [000100], [000156], [000164]) [00086] In order to eliminate or at least reduce such disturbances and reveal the different causal models hidden in the data, an interactive parallel coordinates interface (as shown in FIG. 1C is employed by an embodiment of the Visual Analytics system and method. Via the parallel coordinates, users can directly observe potentially attractive data subdivisions and partition the data by adjusting the brushed value range of variables. Conversely, data partitions can also be detected by the system based on unique values of some variables or as data clusters recognized by clustering algorithms, using the interactive capabilities shown for example, in FIG. 1E.”)
use the selection of variables for fitting the dataset into a first model; (paragraphs [00079]-[00083], [00087], [000111]) and
display a first goodness-of-fit corresponding to the first model (figures 1, 1F, 7, 7A, 12; paragraphs [00025], [00079], [00089[00089] (“ FIG. 1F illustrates the heatmap of the exemplary models, where a darker tile 1 denotes a model with a lower model score (thus better goodness) following the criterion described further hereinbelow in connection with FIGS. 1 and 2 and Equations (l)-(2). FIG. 1B illustrates the causal model denoted by for example, the highlighted tile 10 (that is colored in orange) in FIG. 1F. [00090] In certain aspects or embodiments, in order to find possible groupings of models derived from a dataset, k-medoids clustering is applied which is an effective method in finding the representative objects among all. In the shown example, by setting k = 3 with the controls in FIG. 1F, a new heatmap is generated as shown and described in greater detail hereinbelow in connection with FIG. 7 A.
[00091] By way of background, the set of causal relations between variables of a multidimensional datasets is usually depicted as a Directed Acyclic Graph (DAG) where variables are nodes and a directed edge between two nodes means the first causes the second. In certain aspects or embodiments, algorithms learning the structure of such DAGs can be roughly classified into two categories - score -based algorithms and constraint-based algorithms”]) and
a second goodness-of-fit corresponding to a second model, through the the surface display (figures 1, 1F, 7, 7A, 12; “[000169] The system places the nodes at the same location for each model to facilitate comparisons therebetween for the analyst. In the example shown in FIGS. 7A-7H, the user seeking to use this dataset to relate the unique cycle of the chlorophyll concentration variation with other variables, and hence, the most attractive difference for the user could be that the ChlrConc is associated with other variables differently in the three representative models. Users can also examine other models by clicking on tiles of the heatmap shown in FIG. 7A. Also, the system can cluster models into more groups with controls shown in FIG. 1F, although it is observed that there are indeed three dense areas in the t-SNE layout of these models’ adjacency matrices, as shown in FIG. 7E.).
Muller displays the first goodness-of-fit and second goodness-of-fit though the surface display as above but doesn’t teach, display the dataset through an augmented reality (AR) wearable display and display the first goodness-of-fit and second goodness-of-fit though the AR wearable display.
However, Hubenschmid teaches, display the dataset through an augmented reality (AR) wearable display and display the first goodness-of-fit and second goodness-of-fit though the AR wearable display. ( Fig.1 “STREAM combines spatially-aware tablets with augmented reality head-mounted displays for visual data analysis. Users can interact with 3D visualizations through a multimodal interaction concept, allowing for fluid interaction with the visualizations.”
Section 3.2 right column discloses relation between table display and AR display.
Hubenschmid : “The AR HMD is suited for viewing and interacting with 3D visualizations, thanks to its stereoscopic output and egocentric navigation (cf. [35]). We therefore employ many of its available input modalities (i.e. head-gaze, egocentric navigation) for tasks that require 3D input (e.g. positioning of scatter plots). While the by default available mid-air gestures are also suited for 3D input, we chose to forgo mid-air gestures in favor of concentrating on the tablet interaction. In addition, our choices were also infuenced by the technical restrictions of available hardware (i.e. Microsoft HoloLens 1). Although we used head-gaze in this specifc scenario, our concepts are responsive and work with both head-gaze and eye-gaze, depending on what is available. In contrast to the AR HMD, the spatially-aware tablet excels in viewing and interacting with 2D information, thanks to its high resolution display and touch-based interactions.)
Mueller and Hubenschmid are analogous as they are from the field of data display method using displays.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective fling date of the claimed invention to have modified Mueller to have display the dataset through an augmented reality (AR) wearable display and display the first goodness-of-fit and second goodness-of-fit though the combination of surface display and the AR wearable display as taught by Hubenschmid.
The motivation for the above is to have controlled display in augmented reality and surface display.
Claim 1 is directed to a method and its steps are similar in scope and functions of the elements of the device claim 12 and therefore claim 1 is rejected with same rationales as specified in the rejection of claim 12.
Claim 18 is directed to a non-transitory computer readable media (“[000380] Method examples described herein may be machine or computer-implemented at least in part. Some examples may include a computer-readable medium or machine-readable medium encoded with instructions operable to configure an electronic device to perform methods as described in the above examples. An implementation of such methods may include code, such as microcode, assembly language code, a higher-level language code, or the like. Such code may include computer readable instructions for performing various methods. The code may form portions of computer program products. Further, in an example, the code may be tangibly stored on one or more volatile, non-transitory, or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media may include, but are not limited to, hard disks, removable magnetic disks, removable optical disks (e.g., compact discs and digital video discs), magnetic cassettes, memory cards or sticks, random access memories (RAMs), read only memories (ROMs), and the like”) and its elements are similar in scope and functions of the elements of the device claim 12 and therefore claim 18 is rejected with same rationales as specified in the rejection of claim 12.
Claims 2 and 13, Mueller as modified by Hubenschmid teaches, wherein the second model is based on the plurality of variables. (figures 1, 1F, 7, 7A, 12; “[000169] The system places the nodes at the same location for each model to facilitate comparisons therebetween for the analyst. In the example shown in FIGS. 7A-7H, the user seeking to use this dataset to relate the unique cycle of the chlorophyll concentration variation with other variables, and hence, the most attractive difference for the user could be that the ChlrConc is associated with other variables differently in the three representative models. Users can also examine other models by clicking on tiles of the heatmap shown in FIG. 7A. Also, the system can cluster models into more groups with controls shown in FIG. 1F, although it is observed that there are indeed three dense areas in the t-SNE layout of these models’ adjacency matrices, as shown in FIG. 7E.)
Claims 3 and 14, Mueller as modified by Hubenschmid teaches, wherein the dataset is displayed through the combination of the surface display and the AR wearable display by:
displaying a map through the surface display; (Hubenschmid Fig. 1, 6 the surface display or tablet displays a map or spatially aware content)
displaying layers of data, through the AR wearable display, wherein each layer of data comprises markers corresponding to values associated with a variable of the dataset; (Hubenschmid Fig.1 and 6: AR displays shows multiple layer of graph based on dataset.)
wherein positions of the markers correspond to positions on the map.( Hubenschmid Fig.1 and markers or data corresponds to position of data location in the tablet display.
Section 3.2 right column discloses relation between table display and AR display.
Hubenschmid : “The AR HMD is suited for viewing and interacting with 3D visualizations, thanks to its stereoscopic output and egocentric navigation (cf. [35]). We therefore employ many of its available input modalities (i.e. head-gaze, egocentric navigation) for tasks that require 3D input (e.g. positioning of scatter plots). While the by default available mid-air gestures are also suited for 3D input, we chose to forgo mid-air gestures in favor of concentrating on the tablet interaction. In addition, our choices were also infuenced by the technical restrictions of available hardware (i.e. Microsoft HoloLens 1). Although we used head-gaze in this specifc scenario, our concepts are responsive and work with both head-gaze and eye-gaze, depending on what is available. In contrast to the AR HMD, the spatially-aware tablet excels in viewing and interacting with 2D information, thanks to its highresolution display and touch-based interactions.)
The motivation is same as the motivation of claim 1.
Regarding claim 4, Mueller as modified by Hubenschmid teaches, wherein fitting the dataset into a first model, comprises receiving a selection of terms for the model, and wherein the model is based on a mathematical expression.( Mueller, “ [000130] The Bayesian Information Criterion (BIC) which is applicable to both linear and logistic regressions, serves well in answering the question as to how complex the model should be generated for a given dataset. BIC approach rewards the improvement in fit but may also punish for increasing model complexity. Hence, for a single regression model, BIC is formulated in accordance with Equation (1) provided hereinbelow as:
BIC = -2 ln ?+ k ln(n) Equation (1)
[000131] wherein L is the likelihood of the model, k is the number of independent variables, and n is the number of data points. The BIC of a linear regression can be computed from residuals in accordance with Equation (2) provided hereinbelow as:
[000132] BIC = n ln RSS/n + k ln(n) Equation (2)
[000133] where the residual sum of squares is defined by Equation (2A) provided hereinbelow as: [000134] [AltContent: rect] Equation (2A)
[000135] wherein y is the predicted value of the dependent variable given values of independent variables in a regression equation, and y; is the actual observed value of the dependent variable. The likelihood of logistic regressions can be computed directly using logistic functions. Equation (2) hereinabove also suggests that a smaller BIC score with small residuals and less parameters implies a better regression model.”)
Regarding claim 5, Mueller as modified by Hubenschmid teaches, wherein the first and second models are generalized linear models. (Mueller, Paragraph [130-135))
Regarding claim 6, Mueller as modified by Hubenschmid teaches, wherein the generalized linear models are lineal regression models or logistic regression models. (Mueller, Paragraph [130-135])
Claims 7 and 15, Mueller as modified by Hubenschmid teaches, wherein the first goodness-of-fit corresponding to the first model and the second goodness-of-fit corresponding to the second model are displayed by:
displaying a map through the surface display; (Hubenschmid Fig. 1, 6 the surface display or tablet displays a map or spatially aware content)
displaying a first layer of markers corresponding to the first goodness-of-fit, through the AR wearable display, wherein the first layer of markers corresponds to normalized likelihood values associated with the first model; (Hubenschmid Fig.1 and 6: AR displays shows multiple layer of graph based on dataset.) and
displaying a second layer of markers corresponding to the second goodness-of-fit, through the AR wearable display, wherein the second layer of markers corresponds to normalized likelihood values associated with the second model; wherein positions of the markers correspond to positions on the map.(“ (Hubenschmid Fig.1 and 6: AR displays shows multiple layer of graph based on dataset.)
Mueller teaches first layer and the second layer is a surface display. Hubenschmid is included to display the first layer and second layer in AR displays. Therefore first goodness-of-fit and second goodness-of-fit are displayed through the AR wearable display.)
The motivation is same as the motivation of claim 1.
Claims 8 and 16, Mueller as modified by Hubenschmid teaches, wherein the map is a surface corresponding to geospatial information or to a spatialization of a portion of the dataset. (Mueller, Fig.7E and Paragraph [000165])
Regarding claims 9 and 17, Mueller as modified by Hubenschmid teaches, wherein the markers are rectangular, rectangular prism, circular or spherical markers and wherein the values are normalized and represented as color values or greyscale values. (Mueller. figures 1, 1F, 7, 7A, 12; paragraphs [00025], [00079], [00089]),
Regarding claims 10, Mueller as modified by Hubenschmid teaches, iterating between receiving the selection of variables, fitting the dataset into a first model using the selection of variables and displaying the first and second goodness-of-fit.(Mueller, “[00085] As a result, multiple causal models differing in both structure and regression parameters can arise from such data partitions. Ignoring such facts and always learning the model using the whole dataset will potentially lead to faulty relations returned by inference algorithms. Without data partitioning, the regression model constructed will probably contain considerable large residuals. In addition, the Bayesian Information Criterion (BIC) of a model is computed from such residuals (referring hereinbelow to Equation (2), hence refining these miscalculated causal models based on their score change can also be difficult in these circumstances.
[00086] In order to eliminate or at least reduce such disturbances and reveal the different causal models hidden in the data, an interactive parallel coordinates interface (as shown in FIG. 1C is employed by an embodiment of the Visual Analytics system and method. Via the parallel coordinates, users can directly observe potentially attractive data subdivisions and partition the data by adjusting the brushed value range of variables. Conversely, data partitions can also be detected by the system based on unique values of some variables or as data clusters recognized by clustering algorithms, using the interactive capabilities shown for example, in FIG. 1E.
[00087] These interactive capabilities shown in FIG. 1E also allow users to manage the recognized partitions. Users can save a partition as a tag, recall it in the parallel coordinates by clicking the tag, or fit it to a causal structure by hitting the“Fit Model” button. Most
importantly, one can learn a causal model from each such data subdivision and refine it with the visual approaches as described further hereinbelow in connection with FIGS. 3-4.”)
Regarding claims 11, Mueller as modified by Hubenschmid teaches, wherein the first and second models correspond to models fitted during different iterations. (Mueller, Para[00085-00087], [000111],[000112])
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Tapas Mazumder whose telephone number is (571)270-7466. The examiner can normally be reached M-F 8:00 AM-5:00 PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TAPAS MAZUMDER/Primary Examiner, Art Unit 2615